https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

02/21/2023, 10:04 PM
(totally not related /s) When we make a
VenvPex
it has some guards against the venv being deleted, but if I delete the venv dir I see
path/to/venv_dir/pex: No such file or directory
Working backwards, when we make the PEX with
--venv --seed verbose
I see
pex
in the output JSON having the right path, but the second portion isn't related to any portion in
PEX-INFO
. Where is that second hash coming from?
Doesn't seem like an issue on
main
, so something related to my changes 😕
Ah OK I think I see, pex is likely including the
python
path in the hash, which is absolute and changes per daemon restart 😢
Copy code
Failed to find a compatible PEX_PYTHON=./__toolchain/versions/3.7.16/bin/python.

Examined the following interpreters:
1.) /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8 CPython==3.8.10

No interpreter compatible with the requested constraints was found:

  A distribution for black could not be resolved for /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8.
  Found 1 distribution for black that do not apply:
  1.) The wheel tags for black 22.6.0 are cp37-cp37m-manylinux2014_x86_64, cp37-cp37m-manylinux_2_17_x86_64 which do not match the supported tags of /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8:
  cp38-cp38-manylinux_2_31_x86_64
  ... 598 more ...
So close...
Yeah it really seems like since
sys.executable
is always an absolute path, it's impossible to build a PEX in one sandbox using a relative Python path, then consume it with the venv pex hack in another sandbox as the interpreter hash can never be the same. CC @enough-analyst-54434 am I understanding this right? I'm looking at https://github.com/pantsbuild/pex/blob/main/pex/variables.py#L789
e

enough-analyst-54434

02/22/2023, 4:28 PM
I think so, but this is not really just a Pex problem. If you build a venv using an interpreter and then blow away that interpeter, your venv is broken. This is a well known problem with venvs generally.
I think you'll need a cache of these things that gets symlinked into the sandbox such that the real path to
python
is stable.
b

bitter-ability-32190

02/22/2023, 4:29 PM
In this case, we're not blowing away the interpreter. Just running it from another location 😅
e

enough-analyst-54434

02/22/2023, 4:29 PM
That == blowing away
b

bitter-ability-32190

02/22/2023, 4:29 PM
Don't venvs copy the python binary?
e

enough-analyst-54434

02/22/2023, 4:30 PM
They can and Pex can too. You really, really ought to familiarize yourself with the Pex CLI if you're going to hack on this stuff.
It is not the default though! The python binary is symlinked by default. You can try and find out.
b

bitter-ability-32190

02/22/2023, 4:32 PM
....That's unexpected. Huh
OK yeah guess there's no way to avoid a named cache
All just to say there are known problems with venvs.
b

bitter-ability-32190

02/22/2023, 7:49 PM
OK, unexcitingly switching the entire paradigm to use an absolute path fixes ~everything
2 Views