ambitious-student-81104
04/04/2022, 11:32 PM2.11.0rc0
and switched from a custom constraints file generation script to ./pants generate-lockfiles
(finally!!). However, for VS code users like myself, we still want to generate a constraints/requirement file, something we can run pip install -r
on to install those deps on our local environment. @hundreds-father-404 told me to do ./pants export ::
then <venv>/bin/pip freeze
, and my export
step failed on a problem that has been there for a while, which is our required version of cryptography
no longer releases macOS distributions, but things have been working for us because our production is Linux, and our local installation didn't have a hard requirement on resolving everything (we used pip compile) like ./pants export ::
seems to be strict about.
Is there a way to work around this? We will still be using macOS for local dev for the foreseeable future so the dependency there will still exist. cc @happy-kitchen-89482Dependency on cryptography not satisfied, 1 incompatible candidate found:
1.) cryptography 3.4.8 does not have any compatible artifacts:
<https://files.pythonhosted.org/packages/35/23/b7dc7a019e1a34b024498510686a6b6a400caf6ecd1c0ba102e27dd3658f/cryptography-3.4.8-pp37-pypy37_pp73-win_amd64.whl>
requirers:
via: SecretStorage==3.3.1 -> cryptography>=2.0
via: apache-airflow==2.1.4 -> cryptography>=0.9.3
via: authlib==0.15.2 -> cryptography
via: moto==1.3.14 -> cryptography>=2.3.0
via: paramiko==2.7.2 -> cryptography>=2.5
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> cryptography<4.0.0,>=2.5.0
via: moto==1.3.14 -> sshpubkeys<4.0,>=3.1.0 -> cryptography>=2.1.4
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> azure-storage-blob<13.0.0,>=12.0.0 -> cryptography>=2.1.4
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> pyOpenSSL<21.0.0,>=16.2.0 -> cryptography>=3.2
happy-kitchen-89482
04/05/2022, 3:21 AMlockfile_generator = "pex"
in your pants.toml) then you can use some jq magic to extract enough information to build a venv for the IDE:cat path/to/lockfile.lock | grep -v "^//" | jq -r '.locked_resolves[].locked_requirements[] | "\(.project_name)==\(.version)"'
cryptography
out of that and then run pip install
on itambitious-student-81104
04/06/2022, 7:45 PMhappy-kitchen-89482
04/06/2022, 7:50 PMambitious-student-81104
04/07/2022, 4:27 PMpip compile
to compile all the requirements.txt
into a constraints.txt
, and users can just use pip install -r constraints.txt
locally.Not sure I understand the cryptography issueit's just that some 3rd party packages we depend on have a transitive dep on a package
cryptography
that no longer releases macOS dists.hundreds-father-404
04/07/2022, 4:31 PMambitious-student-81104
04/07/2022, 4:40 PMwhy pants works at all on macOSπ€―
./pants export ::
on master.happy-kitchen-89482
04/07/2022, 4:45 PMambitious-student-81104
04/07/2022, 5:02 PM./pants export ::
on master actually works. Master differs from my feature branch upgrading to pants 2.11 and using pex to resolve in a few ways:
1. master uses 2.10, my feature branch is 2.11.0rc0
2. master is still using a custom pip compile
script to generate the constraints.txt
file, my feature branch is using pex
and ./pants generate-lockfiles
instead, which works, but then ./pants export ::
breaks at the cryptography
errorpants.toml
on cb slackhundreds-father-404
04/07/2022, 5:05 PMhappy-kitchen-89482
04/07/2022, 5:07 PM./pants export ::
works on 2.10 but does not work on 2.11?ambitious-student-81104
04/07/2022, 5:15 PMcryptography
, cryptography-36.0.2-cp36-abi3-macosx_10_10_x86_64.whl
β― pip install -U --no-cache-dir "pyOpenSSL<21.0.0,>=16.2.0"
Requirement already satisfied: pyOpenSSL<21.0.0,>=16.2.0 in /Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/lib/python3.7/site-packages (20.0.1)
Requirement already satisfied: six>=1.5.2 in /Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/lib/python3.7/site-packages (from pyOpenSSL<21.0.0,>=16.2.0) (1.16.0)
Collecting cryptography>=3.2
Downloading cryptography-36.0.2-cp36-abi3-macosx_10_10_x86_64.whl (2.5 MB)
|ββββββββββββββββββββββββββββββββ| 2.5 MB 3.4 MB/s
Requirement already satisfied: cffi>=1.12 in /Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/lib/python3.7/site-packages (from cryptography>=3.2->pyOpenSSL<21.0.0,>=16.2.0) (1.15.0)
Requirement already satisfied: pycparser in /Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/lib/python3.7/site-packages (from cffi>=1.12->cryptography>=3.2->pyOpenSSL<21.0.0,>=16.2.0) (2.21)
Installing collected packages: cryptography
Successfully installed cryptography-36.0.2
WARNING: You are using pip version 21.2.2; however, version 22.0.4 is available.
You should consider upgrading via the '/Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/bin/python -m pip install --upgrade pip' command.
cryptography
that's generated by ./pants generate-lockfiles
is 3.4.8
for some reason
Dependency on cryptography not satisfied, 1 incompatible candidate found:
1.) cryptography 3.4.8 does not have any compatible artifacts:
<https://files.pythonhosted.org/packages/35/23/b7dc7a019e1a34b024498510686a6b6a400caf6ecd1c0ba102e27dd3658f/cryptography-3.4.8-pp37-pypy37_pp73-win_amd64.whl>
requirers:
via: SecretStorage==3.3.1 -> cryptography>=2.0
via: apache-airflow==2.1.4 -> cryptography>=0.9.3
via: authlib==0.15.2 -> cryptography
via: moto==1.3.14 -> cryptography>=2.3.0
via: paramiko==2.7.2 -> cryptography>=2.5
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> cryptography<4.0.0,>=2.5.0
via: moto==1.3.14 -> sshpubkeys<4.0,>=3.1.0 -> cryptography>=2.1.4
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> azure-storage-blob<13.0.0,>=12.0.0 -> cryptography>=2.1.4
via: snowflake-connector-python[secure-local-storage]==2.4.6 -> pyOpenSSL<21.0.0,>=16.2.0 -> cryptography>=3.2
via: ...
s above to repro it?win_amd64.whl
in the pants-generated lockfile?happy-kitchen-89482
04/07/2022, 5:32 PMcryptography
at 3.4.8
is this line in the "via" above: snowflake-connector-python[secure-local-storage]==2.4.6 -> cryptography<4.0.0,>=2.5.0
snowflake-connector-python[secure-local-storage]==2.4.6
requirement, since that is forcing cryptography<4.0.0
ambitious-student-81104
04/07/2022, 9:57 PMcryptography<4.0.0
!pip install -U --no-cache-dir "snowflake-connector-python[secure-local-storage]==2.4.6"
gives me
β― pip show cryptography
Name: cryptography
Version: 3.4.8
Summary: cryptography is a package which provides cryptographic recipes and primitives to Python developers.
Home-page: <https://github.com/pyca/cryptography>
Author: The Python Cryptographic Authority and individual contributors
Author-email: <mailto:cryptography-dev@python.org|cryptography-dev@python.org>
License: BSD or Apache License, Version 2.0
Location: /Users/horus.wu@coinbase.com/.pyenv/versions/3.7.10/lib/python3.7/site-packages
Requires: cffi
Required-by: snowflake-connector-python, pyOpenSSL, azure-storage-blob
happy-kitchen-89482
04/08/2022, 1:12 AM