https://pantsbuild.org/ logo
Powered by
b

bitter-ability-32190

07/06/2022, 4:11 PM
Why do we set PEX_EXTRA_SYS_PATH for running Python instead of PYTHONPATH? Unfortunately this makes Python subprocess harder to run because they don't inherit the path
✅ 1
e

enough-analyst-54434

07/06/2022, 4:35 PM
The 
PYTHONPATH
is scrubbed by the Pex runtime by default. We'd have to set 
PYTHONPATH
plus one of:
Copy code
pex --help-variables
...
PEX_INHERIT_PATH: String (false|prefer|fallback)

    Allow inheriting packages from site-packages, user site-packages
    and the PYTHONPATH. By default, PEX scrubs any non stdlib packages
    from sys.path prior to invoking the application. Using 'prefer'
    causes PEX to shift any non-stdlib packages before the pex
    environment on sys.path and using 'fallback' shifts them after
    instead. Using this option is generally not advised, but can help
    in situations when certain dependencies do not conform to standard
    packaging practices and thus cannot be bundled into PEX files. See
    also PEX_EXTRA_SYS_PATH for how to *add* to the sys.path. Default:
    false.
👀 1
That includes site-packages though, which is not what you want; so it'd be a new / modified feature.
b

bitter-ability-32190

07/06/2022, 4:37 PM
That includes site-packages though, which is not what you want
How Is it not what I want? I want to subprocess run another file with the same set of deps I'm running with.
e

enough-analyst-54434

07/06/2022, 4:37 PM
site-packages is site-packages of the random interpreter on your system, may have system installed packages.
b

bitter-ability-32190

07/06/2022, 4:38 PM
Ah system site-packages. Yeah likely not what I want 🙂
Essentially, I want the behavior of 
PEX_EXTRA_SYS_PATH
to be manifested to 
subprocess.run([sys.exectuable, ...], ...)
calls
How that happens is 🤷‍♂️
e

enough-analyst-54434

07/06/2022, 4:41 PM
And you don't want to / can't (other code), type: 
subprocess.run([sys.executable, ...], env={"PYTHONPATH=os.pathsep.join(sys.path), **os.environ})
?
b

bitter-ability-32190

07/06/2022, 4:41 PM
Bingo
e

enough-analyst-54434

07/06/2022, 4:44 PM
Ok, so maybe an option to have the Pex runtime export PYTHONPATH as such before hand-off to use code. Please file away if that is a feature you want - it seems reasonable to me.
b

bitter-ability-32190

07/06/2022, 4:47 PM
e

enough-analyst-54434

07/06/2022, 4:53 PM
I labeled it as a feature request since I couldn't divine "don't want to" vs "can't" from Bingo. If it's can't and this blocks sane work over in Pants, some additional commentary on the issue would help prioritize. It certainly seems like there would be scenarios where it is a "can't" though even if yours is not one.
b

bitter-ability-32190

07/06/2022, 4:55 PM
It's a "can but my users are throwing bricks and fits because 'this worked in Bazel'".
😅 1
e

enough-analyst-54434

07/06/2022, 5:03 PM
Ah system site-packages.
Even an interpreter found in a random venv will almost always have a site-packages with pip installed for example. So Pex scrubs the site-packages of any interpreter it discovers regardless of its provenance.
It's a "can but my users are throwing bricks and fits because 'this worked in Bazel'".
Ok, that counts as a can't in my book. I'll amend the issue.
✅ 1
Powered by