https://pantsbuild.org/ logo
h

hundreds-father-404

03/09/2022, 4:44 PM
It's surprisingly little code for us to support pex.py installing a lockfile from either Pex's JSON format vs. normal requirements.txt-style. We do still want to change Pants's
generate-lockfiles
to always generate using Pex JSON. But do we think it's valuable to indefinitely let users keep manually generating their requirements.txt style lockfiles, e.g. via pip-compile or poetry export?
Upside: more flexibility for users, e.g. incremental adoption Downside: ā€¢ more code & things to document ā€¢ manual management is a worse experience. It's much harder to get the input requirements right, for example. We can't error when lock is stale, perf will be worse when installing the lock vs using Pex format
b

bitter-ability-32190

03/09/2022, 4:55 PM
I think this question can and should be answered when PEX lockfiles are more mature and stable, no? Until then, I suspect supporting both will be necessary.
h

hundreds-father-404

03/09/2022, 4:57 PM
True. I was asking mostly to scope how robust we need our PEX Json lock vs. requirements.txt-style lock detection to be
b

bitter-ability-32190

03/09/2022, 5:01 PM
(See my GitHub issue comment, we might be able to punt this on the user and get correctness)
h

hundreds-father-404

03/09/2022, 5:03 PM
I'm still writing out my reply, but tl;dr is that manually generated lockfiles have no guarantee that lockfile headers are present šŸ˜• https://pantsbuild.slack.com/archives/C0D7TNJHL/p1646791761839639?thread_ts=1646774475.931779&cid=C0D7TNJHL
b

bitter-ability-32190

03/09/2022, 5:14 PM
Wouldn't we know that they're manually generated though now? through the new option?
h

hundreds-father-404

03/09/2022, 5:16 PM
Ah I don't think it's reliable to rely on reading option settings like that because it presumes that users are regenerating every single lock after they change an option. There is no guarantee of that. You can for example say "I want all new locks to be generated with PEX" via
[python].lockfile_generator
, but not regenerate what you already have. Pants needs to support that imo.
b

bitter-ability-32190

03/09/2022, 5:37 PM
I think thats where the proposed option comes into play, to help bridge discrepancies
h

hundreds-father-404

03/09/2022, 5:40 PM
what do you mean?
b

bitter-ability-32190

03/09/2022, 9:24 PM
(after "offline" discussion) Now that I know there's a way to migrate my pip-style
reqs.txt
into PEX json and keep the same versions (
pex3 lock create --requirements reqs.txt -o foo.lock
) I'm much tolerable to sunsetting all support of pip-style txt file, assuming users are made aware of the migration mechanism and it works as expected.
There's a discussion we're not having of Pants doing the migration for you. Some kind of
./pants generate-lockfiles --migrate
or
./pants migrate-lockfiles
šŸ˜
Could be a plugin provided by Pants, but not enabled by default šŸ¤”
h

hundreds-father-404

03/09/2022, 9:36 PM
I've been trying to figure out why I have a gut feeling of not pushing too hard on a migration script. I think it's because the Poetry lockfiles created atm via
generate-lockfiles
have weird quirks like environment markers. I'm not very confident that we want people using that as the input to regen lockfiles I think it's helpful to offer as an option, but an option that you might not need or want to use
b

bitter-ability-32190

03/10/2022, 2:06 AM
Yeah, I think the main use-case is you weren't using
generate-lockfiles
before, so maybe a solution built into
generate-lockfiles
doesn't make a whole lot of sense šŸ˜…
šŸ‘ 1