I have an extremely ostentatious proposal on how we might shift the paradigm of Pants and user's understanding of dependencies that I'd love feedback on. Ultimately my goal is to fix several issues that get brought up often that stem from the fact that our code makes assumptions about the types of targets in the

dependencies

field, and therefore falls short when presented when unexpected targets. I encourage everyone to read it, reflect on it, and comment with thoughts. It's an ambitious proposal (and will take a community effort to implement if agreed on) so I'm also going to explicitly ask readers to read the whole thing before commenting, and comment from a place of collaboration (and not from a place of challenge). The TL;DR is: 1. shifting from one monolithic

dependencies

field to several more specialized dependencies fields (e.g

file_deps

,

typecheck_deps

) 2. Treating the "file" as the root understanding of most dependencies

I'm struggling to see what the different typed dependency fields adds. if I as a user have said "this depends on this" then ... include those in the sandbox? Isn't the type of the dependency inferred from the target type?

if I as a user have said "this depends on this" then ... include those in the sandbox?

That's at the core of #2 in the TL;DR. Most times the dependency represents file(s) placed in a sandbox, and we should lean into that

Isn't the type of the dependency inferred from the target type?

It's actually more complicated than that, today. The type of the dependency is determined from the type of one of a few key fields on the target type, and possibly their subclass. E.g. since `python_source`'s

source

field is a

PythonSourceField

, rules that care about

PythonSourceField

and

SourceField

can request the source. There's also a secret secondary battle I'm fighting, which is that I might not know or care how a target is consumed when declaring the target. Namely

file

v

resource

. This proposal solves that and takes it a step further.

At the surface, feels like a fair bit more overhead as a user trying to work out how a dependency should be included.

Did you see the part in the proposal where source deps would likely be folded as file deps 😉? And don't forget dep inference is still at play.

yeah. I'll re-read it again.

It's arguably a bit nuanced, and I'm hoping it works in all scenarios, so let me know if/where it breaks down

The problem statement is spot on. resource vs file vs requiring files of a different language have all bit me. Leaning into files to fix materializing sources from disparate targets sounds good. The multiple dependencies fields proposal has a lot of merit. It solves problems we have. But it also increases complexity for both user UX and plugin author UX. I wonder if we could rename/alias

dependencies

to make its target specific behavior clear and then add more deps fields to cover current shortcomings with asset usage. So, for

python_sources

you have the

python_deps

field (renamed/aliased from

dependencies

) and then add

file_deps

, and

resource_deps

. That does not lean all the way into files like you suggested. I’m not sure if that UX differs substantially from what you recommended. Putting python deps (files, or req/package deps) in

files_deps

feels odd, but it would be consistent from target to target - so which aspect of

dependencies

is most important? Consistent naming across target types or the current target-specific behavior (python uses python)?

Yeah that's better I think

Hmm. There are also times where I need to declare that one

python_source

depends on another python file, but as a

file

, not as python code. For example, when the python_source needs to run the dependent python in a subprocess instead of importing it.

That'd be allowed here (I need to update the proposal). In this case the added

file_deps

python_source

wouldn't have a

PYTHONPATH

pointing to it. Which is the right thing to do