Re 2.11 release timeline x Pex lockfiles:

I have no useful opinions about the labels (when branches are cut, etc). I do agree on qualitative timeline though: there will be Python ecosystem gotchas and this will take a while to suss out to something legitimately stable. So as long as we're not fooling people making them think ready by X.

And I'm starting this afternoon on a time-sensitive Toolchain project that will take up most my time 😐 John, I'm thinking to relieve pressure we can have three escape valves: 1. Poetry lockfile generation stays as an option. 2. An option I added last week

[python].resolve_generate_lockfiles

to allow you to manually manage your user lockfiles 3. We continue to not require using

[python].resolves

or

[tool].lockfile

. Not sure tho if we should deprecate

[python].requirement_constraints

Like 2.10 has a soft launch of "multiple resolves", 2.11 has a soft launch of Pex-generated lockfiles.

I leave that all to you and others to decide. I refuse to feel pressure. Pex locks will be solid when they're solid and when I'm working I'm working on them hard.

https://github.com/pantsbuild/pants/issues/13964 summarizes some of this + has thoughts on how to wire up to Pants. fyi @bitter-ability-32190 - several of these tasks are highly parallelizable. tl;dr: • add

[python].lockfile_generator

• detect whether a lockfile is Pex JSON vs requirements.txt-style from Poetry • strip JSON locks of Pants lockfile header (comments & JSON => boom) • update

pex.py

to use

--lock

when appropriate Would you have bandwidth to take on any of those this week? I can try my best to fill in the rest so that we can get this into 2.11.0rc0

this week

@witty-crayon-22786 and I discussed there's also some work needed to stabilize

parametrize

, so rc0 this week is an aspiration and it's possible we slip to next week

I can certainly pick something(s) up this week!

I opened https://github.com/pantsbuild/pants/pull/14740 - this edit is so much smaller than I thought it might be 🙌 @bitter-ability-32190 help appreciated if you have a spare moment to do any of the 3 TODOs in

pex.py

. They're kind of code-golfy ⛳️ It would prob. be good to have a simple unit test for each. I can do the 2 TODOs in

lockfile.py

, along with anything you don't have time or interest in doing in

pex.py

🙂

The 3rd TODO in

pex.py

is highly dubious. Pex absolutely make no guarantees about the format not changing and has a version field in there to allow for it. I'd address that todo last or else spend some time thinking about a sane way to not parse an undocumented format.

The 3rd TODO in pex.py is highly dubious.

It was dubious to begin with...hm. The # of lines in a requirements.txt lockfile is not very helpful because

--hash

throws things off

Pex absolutely make no guarantees about the format not changing and has a version field in there to allow for it.

Good point. Hm.

couldn't the header just add a line about the lock type and then always strip the header

Yeah, but a key safety valve is

[python].invalid_lockfile_behavior

+

[python].resolves_generate_lockfiles

, which let you do manual lockfile generation. We cannot be certain a lockfile header is present in the lock. We probably want to drop that in Pants 2.12 or so, but until redesigned lockfiles are stable stable, this is a safety valve Maybe tho we can assume if you're using a manual lockfile, you're not using Pex format...that makes sense. This all is convoluted 😕

Any concerns about header stripping perf when the operation behind it is a resolve

Agreed

Yeah, lots of complication from releasing a bit too early, but its a complicated project anyway.