https://pantsbuild.org/ logo
Join Slack
Powered by
why would registering a product in `BuildLocalPyth...
# development
a
why would registering a product in 
BuildLocalPythonDistributions.prepare()
cause an export task on a jvm target to suddenly start fetching and compiling things? the only task that makes this product (the link shared libraries task) is set to return immediately on execute, as is BuildLocalPythonDistributions and the compile tasks
w
check 
./pants --explain
before/after
probably introducing new dependencies between goals
was talking with Chris about this earlier today
a
yeah it is
both pants --explain is different and there are now new dependencies between goals
let me get the explain
w
when a goal runs, all tasks in the goal run
which is an annoying part of the v1 model.
a
lol
i'll just call it native-compile
that makes perfect sense and i don't know why it was so confusing before
w
if possible (and it may not be)
...
...you should prefer to let human-called goals take the good names
and then not worry about how many intermediate goals there are
ie, prefer 
compile
to 
native-compile
a
that's what's causing the goal dependency
changing it makes things work
w
on which goal?
can you split the dependency goal instead?
a
confused
w
for example, 
pyprep
has too much stuff in it. it doesn't need to be monolithic
a
ok
i can do that
w
er, wait.
so, the issue is that there is some other task in 
compile
that you don't want to run?
a
zinc is compiling things and running fetches during an export that it did not do anything on before because of adding this goal dependency
this causes test failures
and also timeouts
w
so generally that should be fine, because a graph is not going to contain both kinds of deps
a
well, the one test failure is because we use an intentionally incorrect javac arg
w
and if it does, compiling them both is... ok?
so... is the issue just that there is a bogus testproject?
a
no, it's the tests themselves
the intellij integration test times out because it starts compiling everything
w
you see what i mean though? 
./pants compile $python $java
is ... fine
so then should that test be more specific?
a
maybe? the reason it didn't happen before was because it wasn't compiling anything before
w
OH, i see. this is 
export
having a dep on 
compile
?
a
yes
transitively
because python dist now needs compile
w
gotcha. yea, it shouldn't.
so, think about what the path is there... export doesn't need a dist i don't think... so why is a dist being generated?
basically, what product deps does 
export
have
a
i was going at this from the wrong direction
w
...that would cause compile to run... because export is just supposed to spit out requirements, afaik
cc @wide-energy-11069
a
it requires 
java
and 
scala
if some boolean options are on
w
which is codegen, but not compilation
but, 
export
does depend on 
pyprep
did you introduce a dep from 
pyprep
to 
compile
?
a
yes because build local dists is in pyprep
w
currently 
export
happens before and without 
compile
step
a
build local dists should definitely not be in pyprep
w
@aloof-angle-91616: yea, so... either taking your 
compile.*
task out of the 
compile
goal, or splitting pyprep up a bit more
a
it's easy, build local dists shouldn't have been in pyprep in the first place
w
i'll cross my fingers for you
but now that you've said "it's easy", i fear you're doomed
a
i totally lied, changing the goal didn't fix that
yeah
export requires pyprep because it subclasses ResolveRequirementsTaskBase and that adds an optional product on PythonRequirementLibrary
w
what i was getting at earlier with the "give human-centric goals useful names" is: 
pyprep
is the opposite thing
ie, not for a user to run
so it really shouldn't exist as it does. should just be a bunch of disparate tasks/goals rather than one
if you need to split it, that would be reasonable.
w
related: https://github.com/pantsbuild/pants/issues/3200 
Refactor export task into multiple tasks
a
yeah, but in this case splitting up pyprep won't fix it i think because there's a chain of product requirements that's causing this i think, because export uses that optional python requirements product which goes to build local dists which ends up in the native compile products.
☝️ 1
w
@wide-energy-11069: wouldn't help here, because all tasks in the 
export
goal would run
w
ah ok. not too aware of the context here. but feel like it would be easier to draw this out
a
yeah @wide-energy-11069 if anyone has context it's me and it's not clear to me either lol
i think making a local dists product might be less work than anything else
that would not work at all, i think
w
your point above makes sense. the next question is: what does 
export
truly need? does it actually want compiled binaries? or only 3rdparty requirements?
a
had the exact thought
w
are the compiled binaries masquerading as 3rdparty requirements?
a
i don't think export should have our synthesized requirements at all so as a correctness thing i think this deserves a fix
no build local dists does that already
basically it's representing 
python_dist()
targets as if they're 3rdparty requirements
(i'm pretty sure)
i think i can turn this into a separate change
w
hm, yea... should probably be a new product
@aloof-angle-91616: if for now you introduced a dep from 
export
to "compile python distributions", that would probably be ok. as long as it doesn't also trigger jvm compilation
a
i don't think that should change too much of the code handling it though really just in ResolveRequirementsTaskBase to get both products
w
and then you could split the product in a followup
a
i don't think that would work? there's still going to be a transitive dep from export to compile -- i think changing the goal name to 
native-compile
is the minimal working solution here (since we don't support doing anything at all with native code except using it in python)
w
yes, that is what i was suggesting.
a
ok i thought you were saying somehow short circuit the dep that's great
w
that would still trigger python compilation for export, which isn't ideal, but which is better than also trigger jvm compilation
a
and no it won't trigger jvm compilation
i'll make a followup issue about the dists product
2 Views
Open in Slack
PreviousNext
Powered by