https://pantsbuild.org/ logo
Join Slack
Powered by
How do I specify the python version that Pex uses ...
# general
g
How do I specify the python version that Pex uses while building? I've switched our repo to minimum version 3.9 but some Pex targets still use 3.8 on CI.
Copy code
ProcessExecutionFailure: Process 'Building 9 requirements for ...foo.pex from the locks/pants.lock resolve: ...' failed with exit code 1.
[2023-03-09T21:49:32Z] stdout:
[2023-03-09T21:49:32Z]
[2023-03-09T21:49:32Z] stderr:
[2023-03-09T21:49:32Z] There was 1 error downloading required artifacts:
[2023-03-09T21:49:32Z] 1. emote-rl 23 from git+<https://github.com/EmbarkStudios/emote@ts/memory-logging-proxy-wrapper#egg=emote-rl>
[2023-03-09T21:49:32Z]     pid 2670012 -> /root/.cache/pants/named_caches/pex_root/venvs/eac612bf592de69ec2b624a7a30f7fec79e2c3a3/ddab8011daaee380698ac2fb9701af18c90c03f6/bin/python -sE /root/.cache/pants/named_caches/pex_root/venvs/eac612bf592de69ec2b624a7a30f7fec79e2c3a3/ddab8011daaee380698ac2fb9701af18c90c03f6/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /root/.cache/pants/named_caches/pex_root/pip_cache download --dest /root/.cache/pants/named_caches/pex_root/downloads/resolver_download.tbpzqg62/usr.bin.python3.8 --no-binary :all: --no-deps git+<https://github.com/EmbarkStudios/emote@ts/memory-logging-proxy-wrapper#egg=emote-rl> --index-url <https://pypi.org/simple/> --extra-index-url <https://download.pytorch.org/whl/cpu/> --retries 5 --timeout 15 exited with 1 and STDERR:
[2023-03-09T21:49:32Z] ERROR: Package 'emote-rl' requires a different Python: 3.8.5 not in '>=3.9'
Note that the resolver uses python3.8. I've added the following environment variables:
Copy code
PANTS_PYTHON_INTERPRETER_CONSTRAINTS="['==3.9.*']"
PY="python3.9"
PYTHON_BIN_NAME="python3.9"
and
Copy code
[python]
interpreter_constraints = ["==3.9.*"]
Not sure if this is a caching issue (due to changing version) or something I need to configure. This is on 2.15.0.
h
What do you have under 
[python]
for your lockfile config, and did you regenerate your lockfile?
FWIW you shouldn't need any of those env vars, although it makes sense that you tried those!
g
I have indeed regenerated my lockfile, tons of times! This is what I have right now:
h
Which python is pex itself running on? i.e., what is 
/root/.cache/pants/named_caches/pex_root/venvs/eac612bf592de69ec2b624a7a30f7fec79e2c3a3/ddab8011daaee380698ac2fb9701af18c90c03f6/bin/python --version
?
Presumably 3.8 but let's double-check
Also, try using a more modern pip, to see if that helps:
Copy code
[python]
pip_version = "23.0.1"
(That "requires a different Python" error message is from pip.)
g
Python 3.8.5, indeed. Will try pip version now!
That option doesn't exist in any version of Pants..?
e
@gorgeous-winter-99296 it's in 2.16.0a0 but I don't think that will help. Is there any way to run CI with 
-ldebug
so I can see the command line Pants is using to call Pex here?
g
Sure thing; though I think the command line was above already? Or is that some other one?
h
Oh, sorry, for some reason I thought you mentioned being on 2.16
g
Ah , I couldn't find it in the docs at all. 🙂 No; 2.15. I can try bumping once I've found the commandline @enough-analyst-54434 needs 🙂
e
@gorgeous-winter-99296 that is not the command line Pants uses to call Pex, that is a Pex internal subprocess invocation later on down the line.
g
Copy code
spawned local process as Some(584879) for Process { argv: ["/usr/bin/python3", "./pex", "--tmpdir", ".tmp", "--jobs", "1", "--python-path", "/usr/local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "--output-file", "src.py.erupt.erupt.entry_points/server.pex", "--no-emit-warnings", "--requirements-pex", "local_dists.pex", "--interpreter-constraint", "CPython==3.9.*", "--entry-point", "erupt.entry_points.server", "--sources-directory=source_files", "atomicwrites==1.4.0", "bottle~=0.12.0", "coloredlogs~=15.0", "emote-rl[torch]@ git+<https://github.com/EmbarkStudios/emote@4c5b31753e7a497fa57ab59e13344468510c920c#egg=emote-rl>", "grpcio-health-checking>=1.40.0", "grpcio-reflection>=1.40.0", "grpcio>=1.40.0", "netifaces==0.11.0", "numpy~=1.20", "protobuf~=3.20.1", "ruamel.yaml~=0.16.0", "--lock", "locks/pants.lock", "--no-pypi", "--index=<https://pypi.org/simple/>", "--index=<https://download.pytorch.org/whl/cpu/>", "--manylinux", "manylinux2014", "--layout", "zipapp"], env: {"CPPFLAGS": "", "LANG": "en_US.UTF-8", "LC_ALL": "en_US.UTF-8", "LDFLAGS": "", "PATH": "/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin", "PEX_IGNORE_RCFILES": "true", "PEX_PYTHON_PATH": "/usr/local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "PEX_ROOT": ".cache/pex_root"}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<076529a5e38163646702e6698a95108391763f867f774443d7bcb46abb3c961e>, size_bytes: 417 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, input_files: DirectoryDigest { digest: Digest { hash: Fingerprint<076529a5e38163646702e6698a95108391763f867f774443d7bcb46abb3c961e>, size_bytes: 417 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: {} }, output_files: {RelativePath("src.py.erupt.erupt.entry_points/server.pex")}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 11, description: "Building 11 requirements for src.py.erupt.erupt.entry_points/server.pex from the locks/pants.lock resolve: atomicwrites==1.4.0, bottle~=0.12.0, coloredlogs~=15.0, emote-rl[torch]@ git+<https://github.com/EmbarkStudios/emote@4c5b31753e7a497fa57ab59e13344468510c920c#egg=emote-rl>, grpcio-health-checking>=1.40.0, grpcio-reflection>=1.40.0, grpcio>=1.40.0, netifaces==0.11.0, numpy~=1.20, protobuf~=3.20.1, ruamel.yaml~=0.16.0", level: Info, append_only_caches: {CacheName("pex_root"): RelativePath(".cache/pex_root")}, jdk_home: None, platform: Linux_x86_64, cache_scope: Successful, execution_strategy: Local, remote_cache_speculation_delay: 0ns }
Copy code
ERROR: Package 'emote-rl' requires a different Python: 3.8.5 not in '>=3.9'
e
Yeah, so the 1 VCS requirement. Ok, thanks.
And, last bit. Any chance I could get a copy of your lock file?
Ok, thank you. This is a Pex bug. I'll file shortly and probably have a fix out by morning California time.
g
No rush, I won't look at this again until Monday morning CET ;-)
e
Pretty embarrassing: https://github.com/pantsbuild/pex/issues/2092 This was just pure being dumb. Clearly you're the ~1st person using locks for VCS / local projects @gorgeous-winter-99296 and you're paying a price. Thanks for soldiering through being, what turns out, the tester for this functionality.
Alrighty, this should fix:
Copy code
[pex-cli]
version = "v2.1.129"
known_versions = [
  "v2.1.129|macos_arm64|717388fdf97eb6dad98fbe651debddfd05630aa6ce80557b8430efa9490fb7ec|4082068",
  "v2.1.129|macos_x86_64|717388fdf97eb6dad98fbe651debddfd05630aa6ce80557b8430efa9490fb7ec|4082068",
  "v2.1.129|linux_x86_64|717388fdf97eb6dad98fbe651debddfd05630aa6ce80557b8430efa9490fb7ec|4082068",
  "v2.1.129|linux_arm64|717388fdf97eb6dad98fbe651debddfd05630aa6ce80557b8430efa9490fb7ec|4082068"
]
g
Aweome 🙇 Works perfectly!
e
An interesting thing, the test added for this now fails on main and its due to a PDM release changing the hash of the wheel generated from the VCS requirement: https://github.com/EmbarkStudios/emote/blob/4c5b31753e7a497fa57ab59e13344468510c920c/pyproject.toml#L74-L75 I knew this was an issue already - locks lock dependencies but not build dependencies, but I'm now wondering if there is some sneaky way to solve this. IIUC there is no solution for this existing in the Python ecosystem.
The reason for the hash change is outside of PDM's control, its really a ~bad spec:
Copy code
$ unzip -qc emote_rl-23.0.0-py3-none-any.whl emote_rl-23.0.0.dist-info/WHEEL
Wheel-Version: 1.0
Generator: pdm-pep517 1.1.3
Root-Is-Purelib: True
Tag: py3-none-any
Generator is part of the spec IIRC.
Maybe I can just customize the hash for built wheels and filter out the Generator line.
g
The version is optional per spec;
8. 
Generator
is the name and optionally the version of the software that produced the archive.
e
Hrm. Yeah, It does get in the way of reproducible builds.
If two different version of the build tool output the same bytes, save for that, it would be nice to hash the same.
But it would be better by far for the lock to contain meta-lock info for build tools.
g
Would hashing just the files in the manifest be an option?
e
METADATA is needed too. If Requires-* change, that should be reflected in the hash.
I'm not sure about others. It definitely gets finicky.
I think I'll just have to punt on this for now and update the expected test hash with a comment and wait for the next pdm release to break the test again. Maybe I come up with a solution between now and then.
g
I had a peek at some backends (
hatchling
, 
flit
, 
build
, 
pdm-backend
, 
poetry-core
) and they all put the version in the wheel.
e
Yeah. I think I need to think hard about how to lock PEP-518 build-requires in Pex locks.
5 Views
Open in Slack
PreviousNext
Powered by