https://pantsbuild.org/ logo
Join Slack
Powered by
We're seeing the `pex download` step taking a long...
# general
h
We're seeing the 
pex download
step taking a long time when trying to generate our lock file for our default python resolve. Anything we should go about inspecting? I'm surprised to see it consuming so much cpu. We do have a large number of requirements and are trying to explore where packages might have been inserted that would take a long time, but open to recommendations from those more experienced.
e
Pip is probably backtracking over an sdist that needs to be built to a c-extension. What Pip version are you using?: https://www.pantsbuild.org/docs/reference-python#pip_version Are you trying to lock on a Mac arm?
Pants version is also nice to note as well.
Pants gets in the way a bit here, but if you keep the relevant sandbox you can add --preserve-pip-download-log to the Pex command line. That will output a log path early that you can tail. That usually makes it pretty clear where the backtracking is.
h
We're on linux, pants 2.16.0. That was my concern, too, that some new package was requiring a heavy build time, but it's hard to go track down which one that is.
Sounds like best bet is to recreate the command pants is running with the pex utility ourselves.
e
What Pip version?
h
The default that comes with that distribution. Looks like 
20.3.4-patched
from the docs at least
e
Ok, easy 1st try is fix that. Pip made a lot of improvements to backtracking. Use the latest version available in the docs.
20 is 2020.
It's old.
2 Views
Open in Slack
PreviousNext
Powered by