pex has started to act weird for me today; it's taking an ungodly amount of time (> 5) to resolve requirements for me today, despite having a constraints file in place. I can provide more info and context for sure, but off the top your heads, is there anything in particular that can cause this?

Did this only start after upgrading to rc1?

Yes. But I'm also adding more deps and building more, so I want to make sure that doesn't have something to do with it. Using

-ldebug

shows that it's clearly the pex resolution process that takes long

The only culprit I can think of is adding a lock to avoid a race condition of writing to the Pex cache. But that was in rc0, and the benchmark didn’t show much slowdown. I’m wondering if any of your new deps use sdists, which must be built locally? Those can be particularly slow, up to several minutes :/

constraints.txt

Cool, good find. Although I don’t believe pip uses a sat solver, I think it’s more simplistic than that: first one wins. At least the old resolver, which Pex still uses by vendoring pip (I’m very excited for the new one) It looks like flask-apispec releases a wheel; perhaps one of its dependencies uses an sdist?

oh i thought it might be related to the new resolver, but you're not even using it

Not yet, but we do plan to switch to it

hmmm...this is still a problem, seems to happen whenever i change deps, regardless of whether there's a constraints file configured or not

The constraints file only means that Pants will use a single resolve for everything, whereas otherwise it will use the precise requirements used by your command, like the specific test. Which means if you have 50 tests, each with their 50 different combinations of requirements, you’ll resolve requirements 50 times 😶 The constraints file allows us to avoid that by resolving the whole constraints file, and always using that, even if it’s a superset. If you change any of your requirements, everything has to get re-resolved. But the Pex cache should be used in

~/.cache/pants/named_caches/pex

well, i thought it was that... i have private pypi index configured that i may have had bad connectivity to

Yes, exactly. As you add more requirements, it takes more time for that initial download/install of the dependencies. And some dependencies are much, much slower to install than others (particularly if it’s an sdist that must be built, rather than a prebuilt wheel file)

still it's a pretty frustrating workflow when you're changing your constraints frequently (which i am right now)....obviously i won't always be doing that

It should be caching those in

~/.cache/pants/named_caches/pex

if they didn’t have any changes btw, you can experiment with

[python-setup].resolver_jobs

to improve concurrency: https://www.pantsbuild.org/docs/reference-python-setup#advanced-options The default is 2, but that might be lower than optimal. We set that before the constraints file optimization to only use a single resolve, because it was common to resolve requirements as multiple distinct processes so you could get exhaustion. Generally, now, you only have a single process happening at a time

yeah it looks like it sets it to

$(nproc)

try setting it to

0

for now. This is the code we use:

if python_setup.resolver_jobs:
        argv.extend(["--jobs", str(python_setup.resolver_jobs)])

That’s something we should make less wonky though + revisit the default.

okay thanks, that will help some...i've extracted the pex command now and edited enough to be able to

source pex-cmd 2>&1 | ts > long-pex-run.log

because i want to figure out what's taking it so damn long...i can see built wheels in the cache, it really shouldn't be that unless there's some kinda weird invalidation going on

setting

[pex].verbosity

may also give some insights. I believe we log how log it takes to resolve each req iirc? It takes a value from 0 - 9. Translates to the

-v

option when using Pex directly

could i DM you the results?

for sure