hundreds-father-404
10/20/2020, 8:04 PM[python-setup].resolver_jobs
, which translates to Pex’s --jobs
. It controls Pex’s concurrency level.
We set the default low to 2 because—before we added the single resolve optimization—it was common to build multiple PEXes at the same time. This resulted in exhaustion / an OOM issue for Toolchain, which was tricky to figure out.
Now that we generally only do a single resolve at any given time, I think we can bump it higher. Pex’s default is cpu_count()
. I’m thinking we do cpu_count() / 2
.
But, when you run ./pants package ::
, that may do multiple Pex resolves at the same time. And if you don’t have a constraints file, then this is a bad change. So I’m not sure: optimize for absolute safety vs. better performance?happy-kitchen-89482
10/20/2020, 9:50 PMhundreds-father-404
10/20/2020, 9:51 PMcpu_count() / 2
- not super likely to OOM fwict