busy-ram-14533
11/26/2025, 11:51 PMpex issue as a follow-up from https://pantsbuild.slack.com/archives/C046T6T9U/p1764087004690069
Thanks to your fix, I can now generate a lockfile where I have this pants.toml config
[python.resolves_to_only_binary]
__default__ = [":all:"]
[python.resolves_to_no_binary]
__default__ = ["pyspark"]
However, when I try to package a pex where pyspark is a dependency, I hit this failure:
ProcessExecutionFailure: Process 'Building 3 requirements for subprojects.example.src.python.example.app.pyspark_job/job.pex from the subprojects/example/3rdparty/python/lockfile.jsonc resolve: dd-internal-pyspark>=1.0.0, numpy>=2.3.0, pyspark' failed with exit code 1.
stdout:
stderr:
Failed to resolve compatible artifacts from lock subprojects/example/3rdparty/python/lockfile.jsonc for 1 target:
1. /Users/gregory.fast/.pyenv/versions/3.12.10/bin/python3.12:
Failed to resolve all requirements for cp312-cp312-macosx_15_0_arm64 interpreter at /Users/gregory.fast/.pyenv/versions/3.12.10/bin/python3.12 from subprojects/example/3rdparty/python/lockfile.jsonc:
Configured with:
build: False
only_build: pyspark
use_wheel: True
Dependency on pyspark not satisfied, 1 incompatible candidate found:
1.) pyspark 3.5.2 does not have any compatible artifacts:
<https://depot-read-api-python.us1.ddbuild.io/magicmirror/magicmirror/@current/files/pyspark/3.5.2/sha256%3Abbb36eba09fa24e86e0923d7e7a986041b90c714e11c6aa976f9791fe9edde5e/pyspark-3.5.2.tar.gz>
requirers:
via: pyspark
via: dd-internal-pyspark>=1.0.0 -> pyspark==3.5.2
The artifact URL (from an internal index) is totally valid, that pyspark-3.5.2.tar.gz is there. With only_build: pyspark, I would expect this artifact to be compatible.busy-ram-14533
11/26/2025, 11:52 PMbusy-ram-14533
11/26/2025, 11:58 PM./pex --tmpdir .tmp --jobs 3 --no-emit-warnings --pip-version latest --python-path $'/Users/gregory.fast/.pyenv/versions/3.10.13/bin:/Users/gregory.fast/.pyenv/versions/3.10.15/bin:/Users/gregory.fast/.pyenv/versions/3.11.13/bin:/Users/gregory.fast/.pyenv/versions/3.12.10/bin:/Users/gregory.fast/.pyenv/versions/3.8.12/bin:/Users/gregory.fast/.pyenv/versions/dd-pyspark-check/bin:/Users/gregory.fast/.pyenv/versions/dd-pyspark-test/bin:/Users/gregory.fast/.pyenv/versions/temp_ddpyspark/bin' --output-file subprojects.example.src.python.example.app.pyspark_job/job.pex --emit-warnings $'--check=warn' --include-tools --requirements-pex local_dists.pex --interpreter-constraint $'CPython==3.12.*' --python-path /Users/gregory.fast/.pyenv/versions/3.12.10/bin/python3.12 $'--sources-directory=source_files' $'dd-internal-pyspark>=1.0.0' $'numpy>=2.3.0' pyspark --lock subprojects/example/3rdparty/python/lockfile.jsonc --no-pypi $'--index=<https://depot-read-api-python.us1.ddbuild.io/magicmirror/magicmirror/@current/simple/>' --manylinux manylinux2014 --wheel --no-build $'--only-build=pyspark' --layout zipappbrief-scientist-13682
11/26/2025, 11:59 PMbusy-ram-14533
11/27/2025, 3:04 PMbrief-scientist-13682
11/30/2025, 1:13 AMbrief-scientist-13682
11/30/2025, 2:34 PM--lock and --pylock resolvers. The prior fix got the Pip resolver working and --pex-repository, --venv-repository and --pre-resolved-dists resolvers were always unaffected. So this should be fixed for all scenarios now.busy-ram-14533
11/30/2025, 3:32 PMbusy-ram-14533
12/01/2025, 2:59 PM