🧵 generate-lockfile and uv pip compile hello folks, I just thought I would share a little hack I use for people facing long delays when generating lockfiles or people who run into runtime exceptions even after generating one. The Problem I have a mid-size monorepo with dependencies such as PyTorch, vLLM, TensorFlow etc. I value reproducibility and use

pants generate-lockfiles

but as the repo grows it is taking longer and longer for the command to finish. On top of that even when the resolve finishes, I get PyTorch errors at run time (see thread). Pragmatic solution The pragmatic solution that I came up with is disable

enable_resolves = true

in

pants.toml

and use

uv pip compile --universal

to generate a lock file. Then I use this generated lock file in

python_requirements

. This approach isn't as strict as directly using

pants generate-lockfiles

.

uv pip compile

functionality is a subset of

pants generate-lockfiles

but I made this pragmatic trade off because the reality I face living in Python AI/ML ecosystem. I hope this helps other folks who are prisoners of PyTorch. Questions and feedback I have been doing this for a while and haven't encountered any issues. Maybe a Pants dev will shake their head and say "no no"? What could be a potential drawback of this approach?

I would never shake my head at a user’s pragmatic solution to a user’s problem! If you have a use case we don’t support, the only people we should be shaking our heads at are ourselves…

What are the main negatives of this approach? Lockfile generation is a painpoint for us but I would like to fully understand the tradeoff before switching.

I think Benjy is definitely more qualified to answer this question, but I suspect it's more vulnerable to supply chain attacks? The lock file that

uv

generates doesn't have hashes and it will resolve the version into the concrete binary at run time when it creates the venv. That's what I can think of but I would love to hear what Benjy says.

When you say “use this generated lockfile in python_requirements” I assume you mean you’re using it as a constraints file?

Concretely, I do this:

python_requirements(
    name="requirements",
    source="requirements.lock",
)

Where the

requirements.lock

file is generated by

uv pip compile --universal requirements.txt -o requirements.lock

Ah, so pants still has to run a resolve on those requirements, but it’s a trivial one because every transitive requirement is already pinned in the input requirements

So pants supports reading requirements from a uv lockfile? Is there a reason why pants still uses the pex lockfile format which is much slower?

In this hack Pants is treating the uv “lockfile” as a set of input requirements, which is not the same thing as a lockfile.

Ok I see. So it seems like by using this approach you would not get the positives that are listed under `enable_resolves`:

* Uses --hash to validate that all downloaded files are expected, which reduces the risk of supply chain attacks.
* Enforces that all transitive dependencies are in the lockfile, whereas constraints allow you to leave off dependencies. This ensures your build is more stable and reduces the risk of supply chain attacks.
* Allows you to have multiple lockfiles in your repository.

Well, you could still have multiple of these things, they just aren’t really lockfiles. Pants will still try and resolve them. But the resolve will be trivial and I guess fast.

In the user's perspective, I think it's this we lose: • Uses --hash to validate that all downloaded files are expected, which reduces the risk of supply chain attacks.

uv pip compile --universal

does specify transitive dependencies and Pants already support multiple "universes" of dependencies without having to generate a lockfile...