https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

02/25/2022, 5:38 PM
Do y'all know of any way to speed up building
requirements.pex
? Or rebuilding? (I'm using
run_against_entire_lockfile = true
since we have some big kahuna deps). It takes ~6 minutes on my machine 😭 I'd imagine there's a solution for this which each dep is a little PEX, and then 
requirements.pex
 is simply a pointer to all the little ones. That way: • We can use N processes to create each little PEX (I have 64 cores, let's CRANK it) • We get incremental upgrading when deps change
h

hundreds-father-404

02/25/2022, 5:40 PM
I think that proposal might be similar to https://github.com/pantsbuild/pants/issues/14127
🙌 1
b

busy-vase-39202

02/25/2022, 5:43 PM
64 cores 👀
1
h

hundreds-father-404

02/25/2022, 5:46 PM
We can use N processes to create each little PEX (I have 64 cores, let's CRANK it)
Pex has parallelism. If on 2.10, Pants does it for you. If on 2.9, set https://www.pantsbuild.org/docs/reference-python#section-resolver-jobs
b

bitter-ability-32190

02/25/2022, 5:48 PM
I'm on 2.10.0rc1 🤔
h

happy-kitchen-89482

02/25/2022, 9:52 PM
I think the intention here is to create N pexes in N processes?
We do have performance improvements to resolution in the pipeline
We are working on one big change, and there is also that ticket Eric linked to above, which we may do if the currently in-flight improvements aren't sufficient
w

witty-crayon-22786

02/25/2022, 10:14 PM
mentioned elsewhere, but: https://github.com/pantsbuild/pants/issues/14127#issuecomment-1016887042 is the next step on this currently: essentially, shipping multiple-lockfile support will make both creating and consuming a lockfile faster. it’s possible that
run_against_entire_lockfile
will be less necessary as the followup work for that rolls out (seed-resolve in particular)
👍 1
b

bitter-ability-32190

02/25/2022, 10:20 PM
Ah I see. It's starting to become clearer now (at least the efficient sunsetting)