https://pantsbuild.org/ logo
#general
Title
# general
i

icy-hair-30586

02/25/2022, 9:01 PM
Hi everybody! I’m having a problem generating lockfiles for python, using
./pants generate-lockfiles --resolve=python-default
. I have
ca_certs_path = "/Users/a.../.ca_bundle/certs.pem"
added to pants.toml, but I still get
Copy code
Updating dependencies
Resolving dependencies...

  SSLError

  HTTPSConnectionPool(host='<http://pypi.org|pypi.org>', port=443): Max retries exceeded with url: /pypi/prefect/json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))

  at ~/.cache/pants/named_caches/pex_root/venvs/s/268d4737/venv/lib/python3.7/site-packages/requests/adapters.py:517 in send
I have also set
export REQUESTS_CA_BUNDLE=...
to the same file with the certs, just in case. Is there any other setting I’m missing?
h

hundreds-father-404

02/25/2022, 9:14 PM
Hey Marko! I think this is a limitation of us using Poetry to generate lockfiles -- I don't think it's properly wired up here. There are several limitations with using Poetry, including that it doesn't support
--find-links
, so John has been leading a project for PEX to have lockfile support a la pip. When we can switch to Pex, we will have support or this auth automatically. Related question, are you using
[python-repos]
? In the meantime, some options: 1. Investigate what it would take to get this working with Poetry, keeping in mind it hopefully will go away in 2.11 2. Don't use Poetry to generate the lockfile (i.e.
./pants generate-lockfiles
). You can still use a lockfile by generating however you want and setting it up in
[python].resolves
, then setting
[python].invalid_lockfile_behavior = 'ignore'
i

icy-hair-30586

02/25/2022, 9:19 PM
I’m playing with my first pants project, so i actually think I don’t need this at all. Poetry at least is not involved. I may have gotten lost 🙂 Thanks for the reply!
h

hundreds-father-404

02/25/2022, 9:21 PM
To clarify, when you run
./pants generate-lockfiles
, under-the-hood we are using Poetry to do the lockfile generation. That is changing soon to Pex because of the limitations of Poetry, but isn't ready yet
I’m playing with my first pants project
Ah, if you're just getting started, you might want to skip the
[python].resolves
feature for now. It's only a soft launch in Pants 2.10 because of limitations like what you're experiencing. See
example-python
repo: https://github.com/pantsbuild/example-python/commit/1ba1247b824c8756780477ffd44692a21d702d65
h

happy-kitchen-89482

02/25/2022, 10:50 PM
What would be a good way to generate the lockfile manually, without using Poetry?
h

hundreds-father-404

02/25/2022, 10:51 PM
https://www.pantsbuild.org/docs/python-third-party-dependencies#user-lockfile has a table with several techniques, including the script included in the above link of
generate_constraints.txt
Marko does this make more sense now?
i

icy-hair-30586

02/28/2022, 10:18 AM
I got most of what I wanted to do done for now, and yes, things make more sense. Thanks
I tried a few too many things at the same time so it took a while to get my thoughts in order.
I ended up needing to just add a single custom library to a pre-existing image for Prefect workflow orchestrator, so I just built a wheel and installed it in the image… got me through to the next step I did have trouble running the pex inside the image, even though it ran fine on my machine, but since I didn’t really need to run it in the image, I moved on to other things
h

happy-kitchen-89482

02/28/2022, 3:08 PM
Is your machine a linux machine or a mac?
To build a pex natively on MacOS that will also run in an image requires some settings changes, and also requires that all transitive dependencies be available as prebuilt wheels
i

icy-hair-30586

02/28/2022, 3:30 PM
It's a mac (not m1). Thanks for the tip!
7 Views