https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

08/04/2022, 7:57 PM
@witty-crayon-22786 @enough-analyst-54434 (and others), thoughts on deprecating Poetry as a lockfile generator in 2.14, but with a longer deprecation cycle like don't remove till 2.16?
This pre-supposes that we don't want to support requirements.txt-style lockfiles in general moving forward, given: 1. they aren't secure because they don't lock down the URLs, only the hashes 2. their performance is worse w/ subsetting But that is taking away a possibly valuable option for people. Imo, it's probably fine to take it away. With JVM, we don't give you fallback options
my motivation for this question is that we have bug reports for Poetry like this https://github.com/pantsbuild/pants/issues/15171. I want to know if I can close it as won't fix
w

witty-crayon-22786

08/04/2022, 8:03 PM
my gut says that that would be fine, but doing a survey of open blockers for using PEX lockfiles would probably be good.
h

hundreds-father-404

08/04/2022, 8:06 PM
Note that this contrasts with an earlier idea I floated last month - Poetry 1.2 seems to make a lot of fixes, including allegedly the issue with transitive deps having bad env markers. So we discussed leaning more into Poetry and fixing all the broken windows That still seems viable, although it will always have the performance hit compared to Pex Only: limited dev resources. I suspect our time is better invested improving Pants itself. Maintenance cost would be an issue too, that it's complex having two code paths
but doing a survey of open blockers for using PEX lockfiles would probably
From what I've seen, none that we know of? John did a bunch of fixes for authentication. We do still need to wire up his work on local requirements, but Poetry didn't support that either, so it's not blocking dropping Poetry. Only blocks requiring lockfiles generally
e

enough-analyst-54434

08/04/2022, 8:24 PM
I don't really have an opinion here beyond the usual, I'd like to break users less often than we do with deprecations. As to point 1 above, that is not a good argument. It has to be the case that a sha 256 hash provides most of the security. If that is broken, the whole world falls apart. The fact that an URL would have to be hijacked as well seems basically inconsequential.
h

hundreds-father-404

08/04/2022, 8:32 PM
As to point 1 above, that is not a good argument.
Ack. I didn't think about the limitation that requirements.txt-style lock isn't a true lock until you pointed it out in blog post review 2 months ago. That's fine if it's a weak argument tho
e

enough-analyst-54434

08/04/2022, 8:33 PM
The evidence it's weak is that Poetry has presumably considered things carefully and jettisoned fixed URLs for decent reasons. We all have about that same brainpower, they can't be far wrong.
h

hundreds-father-404

08/04/2022, 8:34 PM
they do have URLs in
poetry.lock
, it's only that
poetry export
is lossy
w

witty-crayon-22786

08/04/2022, 8:35 PM
i didn’t really respond to
This pre-supposes that we don’t want to support requirements.txt-style lockfiles in general moving forward
but: i’m not sure that that follows: it seems reasonable to say “the only built-in lockfile generator is PEX, but you can continue to generate your own lockfile manually for the long tail of weird cases including manual export from poetry”
1
e

enough-analyst-54434

08/04/2022, 8:35 PM
Ah, and us using export is a choice I presume? I don't now if they document their lock file format, but if they do, clearly it can map directly. If we actually wanted to support Poetry users well, we'd directly support Poetry locks, which can have different solutions than Pip.
h

hundreds-father-404

08/04/2022, 8:36 PM
Great point Stu!
e

enough-analyst-54434

08/04/2022, 8:37 PM
But, Stu's point requires we can translate the lockfile to something we can actually use. Right?
I guess you point out export may no longer be broken.
It would be better to directly translate though if the Poetry lock format is stable or better documented, that would make subsetting just as fast.
h

hundreds-father-404

08/04/2022, 8:39 PM
Right now, Pants knows how to consume Pex JSON locks & requirements.txt-style locks with
--hash
. Stu's point means that users can keep producing the latter, only Pants won't produce that style for you directly anymore For example, we know some users use
[python].resolves_generate_lockfiles = false
so that they can manually manage their lockfiles
It would be better to directly translate though if the Poetry lock format is stable or better documented, that would make subsetting just as fast.
That is an option, but:
Only: limited dev resources. I suspect our time is better invested improving Pants itself. Maintenance cost would be an issue too, that it's complex having two code paths
e

enough-analyst-54434

08/04/2022, 8:41 PM
Ok, I find that hollow since we touted supporting Poetry folks, but certainly agreed there are only so many hours.
Supporting requirements.txt with --hash but not generating those for you SGTM.
👍 1
h

hundreds-father-404

08/04/2022, 8:42 PM
I find that hollow since we touted supporting Poetry folks
Well, sort of. We've never supported consuming your
poetry.lock
directly! We support reading your pyproject.toml, and then we gave instructions that you can also manually run
poetry export
Stu's suggestion means that will continue to be true. You can still manually run
poetry export