When I run

generate-lockfiles

twice in row, it still takes 5-10mins and the logs imply that archives are redownloaded and packages recompiled. Shouldn't there be a cache present to avoid this?

The

pip

cache (used under the hood) should be preserved as a named cache. But much like doing

pip install --dry-run

twice, little to none of the resolution is cached

packages shouldn’t be downloaded or recompiled though

And the resolve is 'slow' because of all the index lookups it needs to do, right?

And the resolve is 'slow' because of all the index lookups it needs to do, right?

Well dependency resolution is NP-Complete so the answers for why things are slow dependent to be complex and hinge on the interaction of various heuristics. And the implementation details matter very much in practice. To your original point. Are you seeing

generate-lockfiles

behave slower/differently than

pip

? https://github.com/pantsbuild/pants/issues/21223 also cross links a bunch of breadcrumbs

Well, maybe not strictly the question, but

uv pip compile

resolves the packages in 5-15 seconds.

Yeah, that is not quite the same thing since it isn't a 'universal' lockfile, but

uv

uses a different logarithm implemented in a different language and is generally much faster than

pip

It's true that pip also takes quite some time. However, in our requirements, all versions are pinned via a constraints file. So there is really not much resolving left to do. (We use

uv pip compile

to resolve, then feed that into the lockfile generation via constraints.txt)