https://pantsbuild.org/ logo
Join Slack
Powered by
after further reflection, i think the “tuple of fi...
# development
w
after further reflection, i think the “tuple of file and a metadata source” model is quite clear, and that changes to the graph introspection goals are orthogonal-ish (or at least not an improvement to internal complexity). so: yay!
i do expect that one effect of (file, Target) will be that it no longer makes sense for the target object to “own” files after hydration
h
This is Benjy’s proposal, correct?
For now, it’s a tuple of (Address, optional file name) (Optional to accomadate targets with no source files, like 
python_requirement_library
)
w
mmm, true.
yea, what benjy said there
❤️ 1
h
Cool. I’m reverting generated subtargets right now
w
so, i had a realization about this this morning
the effect of 
(Target, Optional[File])
or 
(Address, Optional[File])
is that it doesn’t address the fact that 
Target
owns zero or more sources right now
i actually do think that 
Address
being either a “for a file” or a “for a target” makes sense
h
is that it doesn’t address the fact that Target owns zero or more sources right now
It addresses in that you would have one pair for each file belonging to a target. If you don’t care about files, we strip those all back down into just the single `Address`/`Target`
w
right, but i don’t think that actually gives you the type safety benefit
because you can use the inner thing accidentally
(…the same goes for Address itself, but with a lower surface area perhaps)
so, i think that how Benjy’s idea connected for me was “yes, it’s a tuple of file and metadata”
but one way to accomplish that is that 
Target
itself could have the optional file.
so a hydrated target would be basically 
(Fields, Optional[Source])
which is roughly the same idea… it’s just that it’s actually target that is the thing that associates metadata with a source.
h
And what is 
Address
? Is your proposal basically “generated subtargets, but always used no matter what”?
w
and an Address continues to have a 1:1 correspondence with a Target: a file address gets you a Target owning the one file, but a target address would get you a Target owning no files, but depending on a bunch of file targets
h
I’m not following what the motivation there is?
w
@hundreds-father-404: yea, basically? when you generate a base target, the effect is that it depends on the “sub/file” targets rather than owning the source
@hundreds-father-404: the motivation is that it is still almost exactly the “tuple of optional file and its metadata” approach
as in, i think this is the same idea, but with a different implementation
h
when you generate a base target
Do you know what BUILD files would look like? If we allow you to still have a target in a BUILD file that has multiple files in its 
sources
, then I think this needs to be flipped. Our starting point is the base target which has multiple files. From there, we can generate the subtargets. I think you’re proposing that then from there, we can generate a new base target that no longer owns sources?
w
but “a hydrated Target always has 1 or zero files” accomplishes the tuple idea in a different way that i think is very, very compatible with existing rules.
@hundreds-father-404: i’m not proposing BUILD file changes… this would be an implementation detail
an advantage of the base target depending on the subtargets is that it “just works” with all existing rules, i think.
depending on the base target is an alias for depending on “all of the files”
h
Okay, then yes, it sounds like we would have the original base target -> generated subtargets -> generated base target. Right now, we have original base target -> sometimes generated subtargets -- I’m not following why we need that new last step? What is the benefit? Generated subtargets already don’t require any changes to rules, minus 
dependees
. My criticisms of generated subtargets are almost entirely based on the user experience of things like file args vs. target args giving a different experience. Afaict, this would still be the case.
w
My criticisms of generated subtargets are almost entirely based on the user experience of things like file args vs. target args giving a different experience. Afaict, this would still be the case.
that part isn’t clear to me actually. it depends what the rules do with the targets.
in particular, if you have an in memory base target which owns no sources, and depends on a bunch of subtargets, what rules would do today would be run each subtarget individually
h
How so? With the current implementation, rules get Targets. They are agnostic to how those were given to it. They could have been explicit targets at 1-1-1 precision; explicit targets at 1 target to 1 file precision; or generated subtargets. The rule doesn’t need to care. It can if it wants to, but doesn’t need to.
w
^ see my last
h
what rules would do today would be run each subtarget individually
This is already the case with generated subtargets, though? I don’t understand how this improves upon the current implementation.
w
ie, you’d get some uniformity because a rule would always run at the file level, unless it wanted to do something else
@hundreds-father-404: i mean that it would happen uniformly, regardless of the CLI specs
because you would always generate the subtargets, and the base target would always depend on them rather than owning its own sources.
h
i mean that it would happen uniformly, regardless of the CLI specs
So, with something like the current 
pytest_runner.py
implementation, you’re saying that no matter what we would always run one-file-at-a-time?
w
with the current implementation, yea.
(so… it would be uniform… for better or worse.)
but i sortof think that that might be the natural consequence of Benjy’s tuple idea
… the rule would have to choose partitions/groups if it wanted them
(which at a basic level might be “group by base target” or something)
cc @happy-kitchen-89482: whole thread
h
it would be uniform
I see. And I acknowledge that uniformity is one of my main criticisms of generated subtargets / file level deps. An alternative, maybe simpler, approach would be to make changes so that file arguments always no matter what use generated subtargets. We could do that if we add the way to disambiguate between multiple owners, e.g. 
@
. Then, with file args, you always get generated subtargets. With target args, you always get normal targets
w
An alternative, maybe simpler, approach would be to make changes so that file arguments always no matter what use generated subtargets.
yes, i think that this should happen… this was one of the things that i think we agreed on in the design session
ie that 
./pants test ::
and 
./pants test '**/*'
would be different, and that that is ok.
h
There are I think two places I find the lack of uniformity confusing: - file args vs. address args give you a different experience - file args sometimes use generated subtargets, but sometimes don’t I think that second part is the more offensive to me.
w
agreed as well.
two is more confusing. i can also see benefits to fixing the first one… what benjy has said about letting rules partition iff they care… sortof makes sense.
h
this was one of the things that i think we agreed on in the design session
Indeed. But we punted on it to avoid inventing this new disambiguation syntax. I think the past two weeks show it may be worth it.
👍 1
Let’s solve this in another thread maybe, but the other major blocker for me is the problem of how we know whether you depend on sibling files in the original base target.
w
oh. sorry. was misunderstanding the question.
yea, elsewhere is fine.
h
Still reading through this, but it sounds like you're proposing that 
target_type(sources=[a, b, c], field1=foo, field2=bar)
acts like a macro that expands to
Copy code
target_type(source=a, field1=foo, field2=bar)
target_type(source=b, field1=foo, field2=bar)
target_type(source=c, field1=foo, field2=bar)
?
More or less, anyway
w
yes.
oh, with one very important tweak
it expands to
Copy code
target_type(dependencies=['a', 'b', 'c'], field1=foo, field2=bar)
target_type(source=a, field1=foo, field2=bar)
target_type(source=b, field1=foo, field2=bar)
target_type(source=c, field1=foo, field2=bar)
ie, the base target “depends on its files, but does not own any” and “each target owns zero or one file”
h
So why do we need the base target at all at this point (after we've expanded it)?
w
@happy-kitchen-89482: because it is possible to depend on the base target. it has an address, and address/target should be 1:1
Open in Slack
PreviousNext
Powered by