wide-zoo-86070
03/16/2022, 1:59 PM[black]
version = 'black==19.10b0'
lockfile = 'black_lockfile.txt'
[python-repos]
indexes = "['<http://xyz.com/artifactory/api/pypi/pypi-abc/simple>', '<http://xyz.fg.rbc.com/artifactory/api/pypi/pypi/simple>']"
However, i see the following error which seems it does not go to customized artifactory.
Process 'Generate lockfile for black' failed with exit code 1.
HTTPSConnectionPool(host='<http://pypi.org|pypi.org>', port=443): Max retries exceeded with url: /pypi/19.10b0/json
curved-television-6568
03/16/2022, 2:06 PMhundreds-father-404
03/16/2022, 4:00 PMgenerate-lockfiles
goal, unfortunately, does not yet work with python-repos
. You can manually generate a lockfile though, as described at https://www.pantsbuild.org/v2.10/docs/python-third-party-dependencies#generate-lockfiles-goal-vs-manual-lockfile-generation
And in Pants 2.11, we add the option to generate lockfiles by using Pex, rather than Poetry under-the-hood. That does work with [python-repos]
. We'll hopefully do 2.11.0rc0 this week, which will add that optionwide-zoo-86070
03/16/2022, 4:01 PMhundreds-father-404
03/16/2022, 4:02 PMwide-zoo-86070
03/16/2022, 4:06 PMgenerate-lockfiles
.hundreds-father-404
03/16/2022, 4:15 PM--hash
to make sure the artifacts you download are what you expect (supply chain attacks)
b) does not enforce that every transitive dependency is pinned; you can leave stuff off
The lockfiles in 2.10 fix those issues and are what lockfiles are actually meant to be.
--
and also i am wondering the (historical) reasons we choose petry instead of pex(pip) forPip does not support generating lockfiles; the best it has is calling
pip freeze
, which is actually what that generate_constraints.sh
script is doing. But that's not a true lockfile, e.g. no --hash
. Also that constraints file might not work on other platforms, like a file generated on macOS might not work on Linux
So, we tried using the popular tool pip-compile
, which generates lockfiles using pip. It worked okay, but also had the same issue of not supporting multiple platforms. In practice, we found this caused lots of issues
Which led us to evaluating pdm and Poetry as alternative lockfile generators. Poetry was the most mature for Pants's use case, but does have several limitations like the [python-repos]
stuff. We decided to go with Poetry because it can handle most use cases, but to at the same time start a project to teach Pex how to generate lockfiles via pip.
Which is where we are today, after some excellent work from John Sirois, Pex can now generate lockfiles! The Python ecosystem is huge and complex, so we're "dogfooding" it and trying to handle some new edge cases, hopefully so that Pants 2.11 can start using Pex for lockfile generation by default
And then the final part of this long story is supporting VCS (git) requirements, which you can't normally lock because it's not a stable thing; the branch might change for example. https://github.com/pantsbuild/pex/issues/1556wide-zoo-86070
03/16/2022, 4:24 PMpip-compile
to generate the lockfiles. But i got the following error, anything I am missing?
InvalidLockfileError: Could not find a Pants metadata block in the lockfile `black_constraints.txt` for `black`. To resolve this error, you will need to regenerate the lockfile by running `./pants generate-lockfiles --resolve={tool_name}`.
hundreds-father-404
03/16/2022, 4:25 PM[python].invalid_lockfile_behavior = 'ignore'
, which you should do if you're manually generating lockfiles