https://pantsbuild.org/ logo
Join Slack
Powered by
(totally not related /s) When we make a `VenvPex` ...
# development
b
(totally not related /s) When we make a 
VenvPex
it has some guards against the venv being deleted, but if I delete the venv dir I see 
path/to/venv_dir/pex: No such file or directory
Working backwards, when we make the PEX with 
--venv --seed verbose
I see 
pex
in the output JSON having the right path, but the second portion isn't related to any portion in 
PEX-INFO
. Where is that second hash coming from?
Doesn't seem like an issue on 
main
, so something related to my changes 😕
Ah OK I think I see, pex is likely including the 
python
path in the hash, which is absolute and changes per daemon restart 😢
Copy code
Failed to find a compatible PEX_PYTHON=./__toolchain/versions/3.7.16/bin/python.

Examined the following interpreters:
1.) /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8 CPython==3.8.10

No interpreter compatible with the requested constraints was found:

  A distribution for black could not be resolved for /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8.
  Found 1 distribution for black that do not apply:
  1.) The wheel tags for black 22.6.0 are cp37-cp37m-manylinux2014_x86_64, cp37-cp37m-manylinux_2_17_x86_64 which do not match the supported tags of /home/josh/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/python3.8:
  cp38-cp38-manylinux_2_31_x86_64
  ... 598 more ...
So close...
Yeah it really seems like since 
sys.executable
is always an absolute path, it's impossible to build a PEX in one sandbox using a relative Python path, then consume it with the venv pex hack in another sandbox as the interpreter hash can never be the same. CC @enough-analyst-54434 am I understanding this right? I'm looking at https://github.com/pantsbuild/pex/blob/main/pex/variables.py#L789
e
I think so, but this is not really just a Pex problem. If you build a venv using an interpreter and then blow away that interpeter, your venv is broken. This is a well known problem with venvs generally.
I think you'll need a cache of these things that gets symlinked into the sandbox such that the real path to 
python
is stable.
b
In this case, we're not blowing away the interpreter. Just running it from another location 😅
e
That == blowing away
b
Don't venvs copy the python binary?
e
They can and Pex can too. You really, really ought to familiarize yourself with the Pex CLI if you're going to hack on this stuff.
It is not the default though! The python binary is symlinked by default. You can try and find out.
b
....That's unexpected. Huh
OK yeah guess there's no way to avoid a named cache
e
All just to say there are known problems with venvs.
b
OK, unexcitingly switching the entire paradigm to use an absolute path fixes ~everything
2 Views
Open in Slack
PreviousNext
Powered by