Hi all, I've been having a strange issue with locking my python requirements for the last few days. Essentially I am encountering

ResolutionImpossible

when running

./pants generate-lock-files

due to multiple pinned versions of bcrypt (3.2.2 vs 4.0.1), but only have a single pinned version (3.2.2) in my top-level requirements. How would I use pants to debug the constraints for each package similar to

pipdeptree

so that I can work out where the issue is coming from?

Hi, have you tried already some of options like

--ldebug

or

--keep-sandboxes=always

mentioned here https://www.pantsbuild.org/docs/troubleshooting

Thanks for the suggestion @refined-addition-53644 - Yep I tried that (exert of output attached) . I can then go into the sandbox and inspect the

pex

and

__run.sh

and could in theory invoke the PEX separately and retry. Interestingly there are no files within the

.tmp

, so for instance I cannot view

/tmp/pants-sandbox-0k4B7q/.tmp/pex-pip-log.mzg5u2ix/pip.log

. How would I go about trying to find the pants dependency or sub-dependency that is causing the issue from here?

I don’t really have an answer but how would you do it without using pants? I feel like this is coming from pip itself. I read a bit about pipdeptree and it looks like it figures out such issues when you already have these packages installed in some global/virtual env.

My first dumb port of call would be to increase the verbosity of

pip install

with the environment variable

PIP_VERBOSE=3

and inspect the output. This is useful for instance when pip is taking a long time to resolve requirements as I can identify situations where pip is attempting to download lots of different versions of packages and testing to see if they satisfy all the package version constraints (in the dependency tree). With pants it is a black box

-ldebug

and/or setting the environment variable

PEX_VERBOSITY=9

doesn't give me any more insight other than knowing that pants is attempting to generate a specific lockfile and the time taken so far to do so - is there a way to get more detail here that I've missed in the pants docs? On the flip side when pip installs everything (which could theoretically include version conflicts which it then warns about), I would see what the dependency tree is for a given virtual environment using deptree and/or pipdeptree which together I can work out where potential version conflicts are coming from. Any help on how to do this sort of debugging using pants would be most appreciated 🙂 🙏

This is all bit beyond my expertise now. I would wait for input from @enough-analyst-54434

@melodic-carpenter-39613 Pex has a

--preserve-pip-download-log

but you won't need that in this case. Your output snippet above shows

... bcrypt==3.2.2; python_version >= "3.6" bcrypt==4.0.1; python_version >= "3.6" ...

in the requirements list Pants invokes Pex with; so the error message is dead on. These 2 conflicting requirements are coming directly from top level user requirements in your repo.

Thanks @refined-addition-53644 for your help 🙂 Hi @enough-analyst-54434, I've tried that and have only one version defined anywhere (

bcrypt==3.2.2

in my

requirements.txt

which feeds into

python-default

resolve). I've came to the conclusion that it is not a top-level dependency that I have specified that is requesting

bcrypt==4.0.1

and causing the conflict and is instead some sub-dependency hence why I am interested in getting more verbose output of the pants locking process.

It is definitely coming from your repo, Pants does not do transitive 3rd party dependency analysis, it uses tools like Pex for that (Coursier for JVM, etc).

Okay good to know that it is an issue within my repo. My requirements file has pinned versions of all the packages feeding into

python-default

, however I do have a

python_requirements

target with a different resolve in a

BUILD

file in the same directory as a pipenv

Pipfile

, however this is not currently causing me issues. How would I filter all targets in

pants dependencies --transitive

to list only

python_requirements

targets, ideally matching

python-default

?

Ok great, before looking at your most recent question, can you confirm that means you now see where the 2 bcrypts are defined in your repo? It sounds like you see the 1 in a requirements.txt and the other in a Pipfile and each is associated with a different resolve?

Nope sorry I didn't explain myself very well, when grepping I only ever found one bcrypt entry in the code base. I only mentioned the other resolve with pipenv

Pipfile

just in case it was relevant, but my main problem is with the default lockfile. Due to the fact I could only find one entry for

bcrypt

in the source code I suspected that the conflict is being pulled in as a subdependency of another one of the top-level python requirements.

No harm, but its super odd since you don't use 3.6 at all!

Ha ha true - I think it must've been put in a long time ago when using python 3.6, but then didn't get updated for python 3.7 (which itself is now incredibly long in the tooth 😆)

Ok, on the grep, I deny your claim. The 2 bcrypt deps must be coming from your repo's source somewhere. Pants has no smarts to divine transitive 3rdparty deps, only Pex does. Its Pants up in your output calling Pex and passing Pex 2 bcrypt reqs.

Gotcha - I'll rerun and see what I get 👀 The lock did previously work, however a number of changes were made recently, including merging in another repo into the monorepo and I was unsure how to effectively debug the root cause / which pants target(s) were to blame. Before the lock failed it was also taking MUCH longer to lock than previously (>5 mins vs 1.5 mins) too and was wondering if there was a way to retain verbose logging from pip.

Can you provide the full content of your prior lock? ALso, what files is the grep -v in your output leaving out? Just Pants generated .lock files or others too?

Sure thing and good shout - of interest is the previously mentioned

Pipfile.lock

. I've included a screengrab of the

BUILD

file in case you can provide some further insight into the behaviour I'll rerun the

./pants --keep-sandboxes=always -ldebug generate-lockfiles --resolve=python-default

now

Ok, so pipenv_requirements reads Pipfile.lock and that is definitely the source of the 2nd bcrypt requirement - I think we agree on that? If so, that means the resolve="radar-routines"

pipenv_requirements

are getting used along with the other requirements where the other bcrypt version is defined to create the lock and it's that that we need to debug.

Ah understood, is it ever possible to use multiple resolves for a target without using the

parametrize()

function?

Whoa - you just skipped and jumped and I plod.

I agree on the 2 reqs as it makes sense, especially as there are quite a few targets across multiple directories (pexes mostly, but maybe one python lib) that were recently merged into the mono-repo and therefore some might be using default lock without meaning to.

Ok, good.

Here is my

pants.toml

with some possibly sensitive stuff removed.

Alrighty then, assuming the

3rdparty/python/BUILD

requirements targets do default to a resolve of

python-default

- which the docs claim they do, then those targets and your

src/python/apps/radar_routines/BUILD

`pipenv_requirements`target do name different resolves. Clearly? those 2 different resolve targets are getting mixed into the process that creates the

python-default

lock and that's why you see 2 bcrypt requirements being passed to Pex in the lock failure snippet you posted. That sounds buggy if so, but I'm also pretty "multiple-resolves at the Pants level" ignorant - I did all the Pex support work to implement locking, but nothing at the Pants layer.

I tried a different resolve, but no joy. I'll continue to play around with pants and post again if I can convince myself it is definitely a bug. In any case, thanks @enough-analyst-54434 for all of your help 🙂 - it was really appreciated as I didn't know where to begin to start unravelling the possible threads and am happier that I understand a little more about how pants works.