https://pantsbuild.org/ logo
Join Slack
Powered by
Silly question here, does anyone know if we're usi...
# development
w
Silly question here, does anyone know if we're using the optimal pex/pip CLI args when generating lockfiles?
I've never looked at that part of the codebase before (yet), but I know there were some enhancements in pip and pex over the past while and I wanted to make sure we were leveraging as many of them as possible. I know 2.21 added 
--no-pre-install-wheels
which was a big win, just wanted to make sure there weren't any other notable flags we missed out on
Annnnd, funnily enough, this update from a week ago highlighted some of what I'm referring to 🙂 https://github.com/pantsbuild/pants/pull/21182
Man, simple example: I used mainline pants, and ensured I was using pip 24.1.x and lockfiles were generated (from cold start) about 50% faster - and that was doing exactly nothing else other than setting the pip version in pants.toml
b
The history of 
src/python/pants/backend/python/goals/lockfile.py
https://github.com/pantsbuild/pants/commits/main/src/python/pants/backend/python/goals/lockfile.py suggests that there's been no recent changes to perf-relevant flags specifically for generating lockfiles, but that obviously doesn't capture global changes like the 
pip_version
default value. Are you in a position to confirm what flags we should be passing for optimal behaviour and update pants to match?
w
Not yet, haven't read through the pants side of it - but I'm getting progressively more familiar with the 
pex
side of it. According to John, fast-deps didn't seem to make a difference a while ago, but I'll be retrying everything. And 24.1 has the hiccup that it EOL's 3.7 as per your GH ticket from a few weeks ago. Right now, just looking for some nice 80/20 wins before we have to go full "rewrite in Rust" with 
uv
which I assume will be a lot bigger 🙂
Looks like 24.2 also has a couple nice, small perf improvements: https://pip.pypa.io/en/stable/news/#v24-2
c
Since pip the project became more willing to merge perf related changes (competition!) the pip version has appeared to be the easiest knob to tune. I have futzed a bunch, but don't recall anything else useful.
w
There is a pex feature for incremental resolves which I wouldn't mind looking at, when I get some free time - whenever that happens 😢
9 Views
Open in Slack
PreviousNext
Powered by