So this is very hacky, but it seems quite doable to convert our repo to uv and then convert uv lockfiles to pants lockfiles Lock times with uv are basically instant and it takes longer to run the conversion script

haha, I was trying to go the other way around and add a new kind of lockfile for uv in pants, but this is a nice dirty hack 👍

I don't know how anything works inside pants tbh, so I have no idea what it would take to support it there, but this was very easy I am mildly concerned that there are edge cases I will eventually stumble on if I do actually use this in practice, but that has more to do with the lack of docs about the lockfile formats, and the fact that they're both meant to be internal

pants uses pex to generate the lockfile, you might find the docs there https://github.com/pex-tool/pex

Nice. Thanks! Do we know what the blocker is to generate lockfiles via uv if any?

I don't really know python packaging well enough to say; it seems to handle the simple case fine, as demonstrated by this conversion. I have not tested anything around multiple python versions at the very least. If you have bandwidth to test the conversion, that would be helpful. We could try and wrap ship this as part of pants where we invoke uv to make it's lockfile and then convert it. I would definitely be happier not being the only user of this code.

The problem is that the python lockfile is tightly coupled with pexes that use it. Every python tool inside pants is a pex, and every pex build is slow. IMO lockfile generation is slow, but it's fine because it's rare, however pex builds are everywhere, so I want to use something like

uv run

instead of building and running pex. However, pants pex logic is quite complicated and there lies the biggest blocker.

I feel like I get more complaints about lockfile generation than execution since lockfile generation is a big tax all at once; and it impacts people very unevenly, folks who are trying new things pay more of a cost Which is just to say, I think solving the lockfile generation problem alone is worth it too

I've created a CI job that gets triggered on dependabot prs and it generates all the changed lockfiles automatically. After that all complaints went away

I started using this in our repo, so far people are happy 🤞

amazing, I'm glad it worked 👍

What's the license?

For the script? Lets go with Apache 2.0 since that's what pants uses, but really it's whatever you need it to be with me disclaiming any formal responsibility:p

Sounds good, thanks

@adorable-psychiatrist-59834 Did you find a way to use your existing requirements here rather than needing a pyproject.toml? Not sure if the pex lockfile requires all the fields I could produce with an alternative command e.g.

uv pip compile

I dont have a pyproject.toml at all

Yes I know, I mean rather than using a pyproject.toml which uv.lock would

pex lock files need URLs & dependencies between your dependencies (& maybe hashes, but not 100% sure) between packages as the minimal thing to actually succeed, so you do need some code to go from a requirements file, even with specific versions to what pex needs

Got most of the way through implementing this i.e.

requirements.txt -> uv pylock.toml -> pex python-default.lock

but found a blocker in that

uv pip compile

does not provide the full

RequiresDists

info for each of the packages its resolving, which Pex needs in its lockfiles. I’m assuming using

uv lock

as OP is provides this correctly. I could get around this by inspecting METADATA files of installed packages or using importlib, but that would necessitate downloading each wheel rather than just remote querying the metadata as

uv

does which, to my understanding, is one of the biggest performance boosters

uv

utilises (alongside the parallel resolution features). I’ve went through the

uv

docs but can’t find anything that would give me the RequiresDists info needed here (see below), but if anybody knows of a possible solution do let me know.

# pex python-default.lock
          "project_name": "sqlalchemy",
          "requires_dists": [
            "aiomysql; extra == \"aiomysql\"",
            "aiosqlite; extra == \"aiosqlite\"",
            "asyncmy!=0.2.4,!=0.2.6,>=0.2.3; extra == \"asyncmy\"",
          ]
...

If you have gotten this far I presume you have seen https://github.com/astral-sh/uv/issues/13032 ?

I have indeed but appreciate the share. Not sure if attempting to utilise this Pex functionality would be too unwieldy.

We only support one python version these days, so I have not run into this

I think that might be my only option aye, just didn’t like the prospect of swapping multiple fields out to use other resolves for each of our pipelines that use different versions.

What's the actual issue with multiple versions? I don't quite understand since I haven't tried

uv doesn’t have an option for multi-version generation of pylock.toml files, which is what I’m using as a middleman file which I think convert to a Pex lockfile, but my interpreters constraints are multi-version. Pex would be able to generate a multi-version lockfile, so I could generate uv lockfiles for each version and write a script to combine them, but it would be less accurate.