Another general question. I have a python-package ...
# general
b
Another general question. I have a python-package A and a service B (which uses A). They both use the same resolve. • since A is a python package that is meant to be used by others, I need non-fixed requirements (e.g. numpy >= 2) • because B is a service, we want the environment to be as stable as possible. The lockfile keeps the environment stable, but from what I understand, if i have to rebuild the lockfile (because of a new package), the versions can change wildly if the requirements are open? Some of the options I considered: 1. have pinned versions in requirements, custom
pyproject.toml
for package A with unpinned dependencies specified in there. But then nothing checks that those are correct, perhaps a custom plugin? 2. have unpinned versions, and somehow figure out a way to minimally update lockfiles. Is there some flag that I have missed? Some common way of doing this?
b
Pex supports this with
pex3 lock update ...
(5 years old) and
pex3 lock sync ...
(2 years old). Pants has been slow to adopt to put it mildly. Pants recently stopped adding invalid comment headers to Pex lock files; so if you're on a new enough version, you might be able to run
pex3 lock ...
commands directly on the lock file to update it. That said, you may still run into issues where you need to update the sibling Pants lock metadata file. On that part I'm really not sure.
h
Optional use of
lock sync
is now implemented in https://github.com/pantsbuild/pants/pull/23158, so this should be in 2.32.0
šŸ™‡ 1
f
You might also just need different
python_requirements()
block and/or a separate resolve. I've been running into kind of a similar thing, we had been using the
<http://requirements.in|requirements.in> -> compile/freeze -> requirements.txt
and then pants lockfiles derived from that .txt with all the frozen versions, but I want to transition to pyproject.toml and uv.lock. pants tailor really insisted on there being a separate python_requirements for the pyprojec.toml and requirements.txt, but since the toml had loose dep constraints, I ended up just creating a separate resolve just for that target. I ended up with something like
Copy code
python_requirements(
    name="reqs0",
    source="requirements.txt",
    module_mapping={
        "grpcio-health-checking": ["grpc_health"],
...
    },
    overrides={
...
        },
    },
) 

...

python_requirements(
    name="reqs1",
    source="pyproject.toml",
    resolve="python-pyproject",
)
we do things a bit...uniquely here though, so take this advice with a grain of salt!
s
I think I've found an alternate solution. put your non-fixed requirements in your project's pyproject.toml use
uv lock
to generate a lockfile with pinned requirements use
uv export --frozen --no-emit-project --no-hashes > uv.lock.txt
to generate a constraints file from the uv lock add this to your `pants.toml`:
Copy code
[python.resolves_to_constraints_file]
python-default = "uv.lock.txt"
(change
python-default
to the name of your python resolve) the run
pants generate-lockfiles --resolve=python-default
to generate a pants lockfile that matches the uv lockfile when you add a new dependency, run these 3 commands again.
uv lock
will ensure that the new dependency (and sub-deps, etc) gets added without auto-updating a whole slew of existing deps in your resolve It's a bit clucky, but i've been using this workaround for a little while now, and it works!
šŸ‘€ 1
f
ā¬†ļø this is brilliant! I actually need this exact thing
s
oh, also note, if any of your deps is a VCS dependency (ex dep in pyproject.toml:
pexpect @ git+<https://github.com/pexpect/pexpect@eb2820cec514c3ed5482e80ad3438cd31f2fa1ef>
), you'll need to filter them out of the constraints file, ie
Copy code
uv export --frozen --no-emit-project --no-hashes | grep -v '.*@.*' > uv.lock.txt
instead of
Copy code
uv export --frozen --no-emit-project --no-hashes > uv.lock.txt
happy to help!