https://pantsbuild.org/ logo
Join Slack
Powered by
Hi, I'm trying to generate a new lock, and it jus...
# general
h
Hi, I'm trying to generate a new lock, and it just takes an infinite amount of time (it used to be 25-35 minutes).
Copy code
> pants generate-lockfiles --resolve=python-default
โ ‚ 4795.88s Generate lockfile for python-default
Is there anything I can do to debug it? There is probably a bad constraint that wrack havoc, but I don't know which and there are 370 dependencies in the repo...
๐Ÿงต
h
Which Pants version is this? In Pants 2.16 (currently only available in dev releases) you can set a newer pip version, which may have resolve speedups.
Also, and possibly more importantly, what are your interpreter constraints?
Pants will resolve for all compatible interpreter versions, which may be completely unnecessary for you.
h
I'll try with latest pip and report back. The Pants version is 
2.16.0.dev3
. The interpreter constraints are 
["CPython>=3.9,<3.11"]
. I'd actually want to include 3.11 as well, but I'll wait with that since I still tackle other challenges. Also, thanks for the quick reply ๐Ÿ™‚
BTW, even if the newer pip is twice as fast, it won't be of much help since it is already at 
8487.55s Generate lockfile for python-default
. So is there no way to debug it currently?
h
What happens if you pin this to 
CPython==3.9.13
or something like that (whatever you have on your system)
The newer pip might avoid backtracking holes in some cases, so it could be many times faster
It's potentially faster due to changes in the resolution algorithm
e
I'd scrutinize that IC question and answer more closely. See: https://github.com/pantsbuild/pants/issues/17978
h
Trying with the interpreter pinned and 
macos_big_sur_compatibility = true
(IDK what 
macos_big_sur_compatibility
is and I didn't find documentation for it in the docs). Is there a documentation on how to change the pip version?
h
./pants help-advanced python
and look for pip_version
And per the issue John linked to, apparently you must set the ICs via 
[python.resolves_to_interpreter_constraints]
h
Changing it to
Copy code
[python]
interpreter_constraints = ["CPython==3.9.13"]
Made it complete it pretty fast. So now I'm confused due to the 
python.resolves_to_interpreter_constraints
thing. Also, I do need it to be resolved for 3.10 and 3.11 as well...
Could it be that the range causes it to be resolved for multiple patch versions of the same minor? (that is 
3.9.1
, 
3.9.2
, ...)
Also, I must say that 
pants help-advanced python
is really nice in terms of docs as well as the fact that it show the current value and where it comes from! ๐Ÿš€
h
Huh, this would seem to be in contradiction with https://github.com/pantsbuild/pants/issues/17978, although that mentions 
dev4
and you're on 
dev3
, so maybe the issue is newly introduced.
But anyway, there is your answer - resolving for many interpreters takes a long time. Usually distributions will support an entire minor version, so it's not actually resolving for every patch version separately. But in some cases there are pathologies, e.g., distributions that declare that they don't support 3.9.0, specifically, because of an issue with that first minor release.
You could look at the pip log and see if that gives you information on what it's having trouble with, and exclude those interpreter versions.
Or you can try limiting like 
interpreter_constraints = ["CPython==3.9.13", "CPython==3.10.3']
, or still use ranges, but finer ranges
h
Thanks! Also, this just came in: I've reverted the changes and changed only the pip version to 
22.3
and pip stopped with an error, rather than continuing into infinity. I'll resolve it and hopefully it will let me lock afterwards. Will report back if I find anything interesting (though it won't be today). Thanks for everything ๐Ÿ™‚
So reporting back: Changing the Pip version to 22.3 in 
pants.toml
made the resolution much faster as well as flagging the underlying issue (which didn't happen with the default pip version set in Pants 2.16.0.dev3). After I've fixed the underlying issue (which was in a package from another repo), the locking completed successfully and about as twice as fast as it was before. My TL;DR is that I recommend setting 
22.3
as the default version of Pip to be used by Pants, and barring that, at least add it to the documentation.
5 Views
Open in Slack
PreviousNext
Powered by