stocky-helmet-2265503/21/2023, 5:44 PM
and I have tests that I want to run against this whl. I setup 2 resolves, one for building the package and one for running the tests. The tests resolve depends on the generated whl by setting
, and this works if the whl has already been made. Pants doesn’t seem to infer that it should always package the whl before running the tests, and I don’t see a way to tell pants that it should other than making a custom target. I can do both steps manually in my CI, but that to me defeats the purpose of using pants. Is there an easy way to do this?
python-repos.find_links = ["file://%(buildroot)s/dist"]
witty-crayon-2278603/21/2023, 5:55 PM
stocky-helmet-2265503/21/2023, 5:57 PM
witty-crayon-2278603/21/2023, 5:58 PM
target in the
field of a
target will cause it to be loaded as if it had been pip installed, iirc.
stocky-helmet-2265503/21/2023, 6:00 PM
is in the same resolve as the
used by the
, what tells the
to not infer a dependency on the
target ignore the `python_distribution`’s dependencies?
witty-crayon-2278603/21/2023, 6:01 PM
happy-kitchen-8948203/21/2023, 6:29 PM
is interpreted as "build the dist and consume the relevant code from it, instead of from the underlying sources". Under the hood, python subtracts the sources from the dists from the direct sources.
stocky-helmet-2265503/21/2023, 7:30 PM
. It appears to still find the files - I don’t know if by source or by whl - but I’m getting a
. Per the stacktrace a test imports code in the whl which imports code in another whl which tries to import code in another whl and that last one is not found. When I ran the tests with a direct dependency on 3rd party dependencies it ran fine, but not running against the whl.
I don’t see a way to add modules or module_mapping in python_tests
happy-kitchen-8948203/21/2023, 9:04 PM
but just regular
stocky-helmet-2265503/22/2023, 3:27 AM
, though using that brings up another unfortunate issue….More background context here but I’ll try to summarize: my project has different 3rd party dependency versions depending on whether python 3.7 is used or python 3.10. I tried to solve this with environment markers, but Pants does not seem to like those so I used separate resolves in addition. I’m using a
which depends on multiple resolves to make a whl that works whether you use python 3.7 or 3.10 (through environment markers) that all seems to work up until where we are now - if I have the
depend on the
, it complains saying the target uses one resolve but has dependencies in the other resolve and throws
. Conceptually this error does make sense, but it makes me believe either this can’t be done - or I’m doing something wrong along the way..
happy-kitchen-8948203/22/2023, 1:53 PM
stocky-helmet-2265503/22/2023, 2:23 PM
Thanks for the extra detail. Can you explain why you want A to depend on “the output of B” rather than just on B’s sources?From what I’m told, doing it this way can help prevent some strange types of errors particularly with extension modules
The idiomatic way here would be for A to depend on B from source when you run tests, even though you also publish BThis is what I’m used to doing - the requirement is coming from others and I’ll admit I don’t fully understand it - hopefully the above response makes sense
Re environment markers, I would have expected those to work, can you elaborate on what error you were seeing with them? And also, are you using a lockfile?I am using a lockfile - I have a demo of this not working here: https://github.com/nikwotton/Broken-Pants-Demo/tree/broken-lockfiles I agree it should work and pants’ documentation suggests it would, but trying to generate the lockfile fails. Note it has the same failure whether the requirements are in requirements.txt or in
happy-kitchen-8948203/22/2023, 8:05 PM
, does it?)
stocky-helmet-2265503/22/2023, 8:32 PM
and yes, if B contains extensions then you would need to represent it as aThis specific package I’m working on does not, but I’m working on getting this setup in such a way that it can be a blueprint for adding pants to all packages in the monorepo so I want to set this up as if it did, does it?
Yeah, I think filing an issue would be the good next stephttps://github.com/pantsbuild/pants/issues/18555
enough-analyst-5443403/23/2023, 2:21 AM
that contains the split requirement:
This is unlockable in a single resolve - Pex will never be able to lock something like this in a universal lock. Pip fundamentally does 1 resolve and that set of requirements is unsolvable since the req range is disjoint. So this forces 2 resolves. The issue there is Pants needs to be able to allow creating a single
numpy >=1.16.2,<1.20; python_version == '3.7'
numpy >=1.21.3,<1.22.0; python_version == '3.10'
that combines requirements strings from 2 resolves then. It doesn't even come close to allowing this today.
stocky-helmet-2265503/28/2023, 7:57 PM
I put a comment on the GitHub issue though, not sure if you saw it - testing against the wheel even within a single resolve seems to run into issues. I don’t understand the error message (linked here), but it seems like running a test against a
which has any 3rd party dependencies cannot be done as of now - hoping I’m just doing something wrong