https://pantsbuild.org/ logo
Join Slack
Powered by
Questions on `generate-lockfiles` . I am seeing ge...
# general
m
Questions on 
generate-lockfiles
. I am seeing generating lockfiles for our monorepo taking like ~1h. I understand that the problem isn't easy because fundamentally it is a NP-complete problem. I am curious of how the resolution works and where the bottleneck is. Is the resolution implemented using PubGrub algorithm in Rust like this? https://github.com/pubgrub-rs/pubgrub Or is the bottleneck on network calls to download the dependency metadata? What are people's thoughts on speeding this up?
w
Can of worms, not a single simple answer: https://github.com/pantsbuild/pants/issues/21223 Check out the links to pex and to pip about this, there is also discussion about seeing whether UV could be used instead
🙏 1
m
Ok, it seems this is a can of worms... but reading between the lines it is 
pip
resolve that is a large bottleneck... that sucks...
w
Yeah, there is a ton of legacy in place, but yeah, if pip can resolve a bunch of outstanding tickets (which have been worked on, and have some progress), there should be some updates on that front. Also, in your case, ensuring you’re using the latest pip/pex is also a good start, in case any updates have happened in the last little while
m
Ok, but how do I update to use the latest pip/pex? I thought that is dependent on Pants version...
w
Defaults are: https://www.pantsbuild.org/prerelease/reference/subsystems/pex-cli#version You can search through the slack to see how these get updated - just to ensure regardless of your pants version, you’re using the newest pex
m
w
Yeah might be that one - we have it a lot in the slack chat
m
Ah... ok thank you so much for the pointers... I really appreciate it...
👍 1
For those who stumble across this thread. I can confirm that bumping pip and pex dramatically shortened the resolution time for us. >1h -> 15min. This is the relevant config in `pants.toml`:
Copy code
[python]
enable_resolves = true
pip_version = "latest"

[pex-cli]
version = "v2.31.0"
known_versions = [
  "v2.31.0|macos_arm64|abd5c7c3f3f21165092aca31a6d5a9559544dc61e7aaf7839d1c86f4ed8645bd|4376378",
  "v2.31.0|linux_x86_64|abd5c7c3f3f21165092aca31a6d5a9559544dc61e7aaf7839d1c86f4ed8645bd|4376378",
]
w
🎉
If you bump it up to 2.32, you might be able to get some Pip 25 enhancements too
12 Views
Open in Slack
PreviousNext
Powered by