So I'm trying to build a lockfile for a resolve pinned to Python 3.12 but I'm getting the following error:

pex: Resolving for:
  cp312-cp312-macosx_14_0_arm64 interpreter at /Users/nicholas.dellosa/.pyenv/versions/3.12.4/bin/python3.12: 56544.1ms
pid 4206 -> /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/bin/python -sE /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache --log /private/var/folders/v7/qb819xs91_55px65swlk4nqr0000gp/T/pants-sandbox-VtNiyu/.tmp/pex-pip-log.1s5we27x/pip.log download --dest /private/var/folders/v7/qb819xs91_55px65swlk4nqr0000gp/T/pants-sandbox-VtNiyu/.tmp/tmp6k4gb2sl/Users.nicholas.dellosa..pyenv.versions.3.12.4.bin.python3.12 awscli beautifulsoup4 boltons boto3 configparser==3.8.1 createsend data-diff==0.7.10 databricks-cli datadog docutils==0.14 elasticsearch>=8 en-core-web-sm@ <https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz> factory-boy findspark fuzzywuzzy gcsfs==2024.6.1 geopandas geopy google-api-core<3.0.0dev google-api-python-client google-cloud-aiplatform google-cloud-bigquery google-cloud-bigtable google-cloud-firestore google-cloud-kms<3,>=2 google-cloud-pubsub google-cloud-recommendations-ai google-cloud-retail google-cloud-secret-manager google-cloud-spanner google-cloud-storage>=1.25.0 google-oauth ipykernel ipython==8.12.2 jinja2==3.1.2 langchain librato-metrics mleap mlflow mock-firestore nltk numpy<2,>=1.26.4 oauth2client openai==1.40.1 opencv-contrib-python>=4.2.0 pandas==1.3.5 pandas_gbq<0.18 parso pendulum pinecone-client==3.0.2 plotly prompt_toolkit protobuf==3.19.5 psycopg2-binary pure-sasl>=0.6.2 pydantic==2.5.2 pyspark[sql]==3.1.3 pytest-cov pytest-flakes pytest-mock pytest-raises pytest<7 python-Levenshtein pytz rake-nltk requests requests-mock scikit-learn setuptools shapely<2 simplejson spacy==2.3.9 spark-nlp-display spark-nlp==4.2.6 sqlalchemy==1.4.49 sqlparse stringcase testcontainers typing-extensions user-agents validators~=0.22.0 xgboost==1.7.4 yake zi_api_auth_client --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
pip: Ignoring the following environment variables in Pex venv mode:
pip: _PEX_CACHE_ACCESS_LOCK=0|3|/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/access.lck
pip:   error: subprocess-exited-with-error
pip:   
pip:   × Getting requirements to build wheel did not run successfully.
pip:   │ exit code: 1
pip:   ╰─> See above for output.
pip:   
pip:   note: This error originates from a subprocess, and is likely not a problem with pip.
pip: error: subprocess-exited-with-error
pip: 
pip: × Getting requirements to build wheel did not run successfully.
pip: │ exit code: 1
pip: ╰─> See above for output.
pip: 
pip: note: This error originates from a subprocess, and is likely not a problem with pip.
pip:    Running command Getting requirements to build wheel
pip:    Traceback (most recent call last):
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
pip:        main()
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
pip:        json_out['return_val'] = hook(**hook_input['kwargs'])
pip:                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
pip:        backend = _build_backend()
pip:                  ^^^^^^^^^^^^^^^^
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/venvs/0/324f028a3aafef516c385157da7e1edb6f0177f5/52fa0a13a5190f9a81a9471ef1516ea9a788dcde/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
pip:        obj = import_module(mod_path)
pip:              ^^^^^^^^^^^^^^^^^^^^^^^
pip:      File "/Users/nicholas.dellosa/.pyenv/versions/3.12.4/lib/python3.12/importlib/__init__.py", line 90, in import_module
pip:        return _bootstrap._gcd_import(name[level:], package, level)
pip:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pip:      File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
pip:      File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
pip:      File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
pip:      File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
pip:      File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
pip:      File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
pip:      File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
pip:      File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
pip:      File "<frozen importlib._bootstrap_external>", line 995, in exec_module
pip:      File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-build-env-oibl21gb/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
pip:        import setuptools.version
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-build-env-oibl21gb/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
pip:        import pkg_resources
pip:      File "/Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-build-env-oibl21gb/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2191, in <module>
pip:        register_finder(pkgutil.ImpImporter, find_on_path)
pip:                        ^^^^^^^^^^^^^^^^^^^
pip:    AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
pip:    ERROR: Getting requirements to build wheel exited with 1

I understand the error, and it seems pretty common. However, I can't determine which dependency is causing it. Is there any flag or something I can enable to figure out which requirement is causing this issue? I'm assuming it's a requirement and not a pex issue because I'm using pex 2.19.1.

Does the pip log contain any further information? That would be this file:

/private/var/folders/v7/qb819xs91_55px65swlk4nqr0000gp/T/pants-sandbox-VtNiyu/.tmp/pex-pip-log.1s5we27x/pip.log

Okay so I preserved the sandbox and added pex's

--pip-log

option and was able to figure out that the issue was related to the

shapely<2

requirement. However,

shapely<2

only supports up to Python 3.10, so I'm surprised the issue wasn't caused earlier.

What I see in your output above is:

numpy<2,>=1.26.4

I’m wondering if that range needs to be tightened a bit further in the requirements

1.26.4 should be fine as a minimum, but it looks like it's collecting older versions as well.

2024-10-08T09:23:27,580 Collecting numpy<2,>=1.26.4
2024-10-08T09:23:27,580   Obtaining dependency information for numpy<2,>=1.26.4 from <https://files.pythonhosted.org/packages/76/8c/2ba3902e1a0fc1c74962ea9bb33a534bb05984ad7ff9515bf8d07527cadd/numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl.metadata>
2024-10-08T09:23:27,580   Created temporary directory: /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-unpack-4gpxhcmb
2024-10-08T09:23:27,580   Looking up "<https://files.pythonhosted.org/packages/76/8c/2ba3902e1a0fc1c74962ea9bb33a534bb05984ad7ff9515bf8d07527cadd/numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl.metadata>" in the cache
2024-10-08T09:23:27,581   Current age based on date: 1708218
2024-10-08T09:23:27,581   Ignoring unknown cache-control directive: immutable
2024-10-08T09:23:27,581   Freshness lifetime from max-age: 365000000
2024-10-08T09:23:27,581   The response is "fresh", returning cached response
2024-10-08T09:23:27,581   365000000 > 1708218
2024-10-08T09:23:27,581   Using cached numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl.metadata (61 kB)
2024-10-08T09:23:27,581   Created temporary directory: /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-metadata-t24hypby

Then when it collects pandas

2024-10-08T09:23:29,294   Collecting numpy==1.26.2 (from oldest-supported-numpy>=0.10)
2024-10-08T09:23:29,294     Obtaining dependency information for numpy==1.26.2 from <https://files.pythonhosted.org/packages/2a/17/1fdc154e75d24d8c20c42b71bae1b5cf752453f0fc3a2504bbb810293dd1/numpy-1.26.2-cp312-cp312-macosx_11_0_arm64.whl.metadata>
2024-10-08T09:23:29,295     Using cached numpy-1.26.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (61 kB)

Then when it collects thinc

2024-10-08T09:24:11,299     Collecting numpy>=1.15.0
2024-10-08T09:24:11,300       Using cached numpy-2.1.2-cp312-cp312-macosx_14_0_arm64.whl.metadata (60 kB)

Does the pip log say anything? I usually see this happen because it gets stuck trying to resolve some really wide constraints for certain packages

This is the only smoking gun I see

2024-10-08T13:19:30,196 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See <https://pip.pypa.io/warnings/backtracking> for guidance. If you want to abort this run, press Ctrl + C.
2024-10-08T13:19:30,196 Will try a different candidate, due to conflict:
2024-10-08T13:19:30,196     The user requested protobuf~=3.19
2024-10-08T13:19:30,196     google-api-core 2.20.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0.dev0 and >=3.19.5
2024-10-08T13:19:30,196     google-cloud-aiplatform 1.69.0 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-bigquery-storage 2.26.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-bigtable 2.26.0 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-firestore 2.19.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-kms 2.24.2 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-pubsub 2.25.2 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-recommendations-ai 0.10.12 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-retail 1.21.2 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-secret-manager 2.20.2 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-cloud-spanner 3.49.1 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     google-api-core[grpc] 2.20.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0.dev0 and >=3.19.5
2024-10-08T13:19:30,196     google-cloud-resource-manager 1.12.5 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     googleapis-common-protos 1.65.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0.dev0 and >=3.20.2
2024-10-08T13:19:30,196     grpc-google-iam-v1 0.13.1 depends on protobuf!=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <6.0.0dev and >=3.20.2
2024-10-08T13:19:30,196     grpcio-status 1.62.3 depends on protobuf>=4.21.6

Hmm yeah, that may well be the problem. Is that the last line in the logs?

Nope, the most recent is

2024-10-08T14:07:31,895 Collecting google-cloud-bigquery-storage
2024-10-08T14:07:31,895   Obtaining dependency information for google-cloud-bigquery-storage from <https://files.pythonhosted.org/packages/32/56/e4162ca337f4012a41eb04db80560c9424024e1c15b98176ca6922a097d0/google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl.metadata>
2024-10-08T14:07:31,895   Created temporary directory: /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-unpack-a1d6v79e
2024-10-08T14:07:31,895   Looking up "<https://files.pythonhosted.org/packages/32/56/e4162ca337f4012a41eb04db80560c9424024e1c15b98176ca6922a097d0/google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl.metadata>" in the cache
2024-10-08T14:07:31,896   Current age based on date: 1634262
2024-10-08T14:07:31,896   Ignoring unknown cache-control directive: immutable
2024-10-08T14:07:31,896   Freshness lifetime from max-age: 365000000
2024-10-08T14:07:31,896   The response is "fresh", returning cached response
2024-10-08T14:07:31,896   365000000 > 1634262
2024-10-08T14:07:31,896   Using cached google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl.metadata (5.6 kB)
2024-10-08T14:07:31,897   Created temporary directory: /Users/nicholas.dellosa/.cache/pants/named_caches/pex_root/pip/0/24.0/pip_cache/.tmp/pip-metadata-lfy_rrh1

when I had these types of issues before, I used

uv pip install -r requirements.txt

. It usually returns within a few seconds if it can't figure out how to satisfy the dependencies. Unsure if that's an option for you here or not...

Is there a way to switch pex over to using

uv pip

? I know there's been discussions about it long-term.

Yeah, usually the issue is that there’s one dependency with really hard-to-satisfy constraints that is slowing down the resolution. Glad you found it quickly 🙂

how do you preserve the pip log ?