Why do we set PEX_EXTRA_SYS_PATH for running Python instead of PYTHONPATH? Unfortunately this makes Python subprocess harder to run because they don't inherit the path

The

PYTHONPATH

is scrubbed by the Pex runtime by default. We'd have to set

PYTHONPATH

plus one of:

pex --help-variables
...
PEX_INHERIT_PATH: String (false|prefer|fallback)

    Allow inheriting packages from site-packages, user site-packages
    and the PYTHONPATH. By default, PEX scrubs any non stdlib packages
    from sys.path prior to invoking the application. Using 'prefer'
    causes PEX to shift any non-stdlib packages before the pex
    environment on sys.path and using 'fallback' shifts them after
    instead. Using this option is generally not advised, but can help
    in situations when certain dependencies do not conform to standard
    packaging practices and thus cannot be bundled into PEX files. See
    also PEX_EXTRA_SYS_PATH for how to *add* to the sys.path. Default:
    false.

That includes site-packages though, which is not what you want

How Is it not what I want? I want to subprocess run another file with the same set of deps I'm running with.

site-packages is site-packages of the random interpreter on your system, may have system installed packages.

Ah system site-packages. Yeah likely not what I want 🙂

And you don't want to / can't (other code), type:

subprocess.run([sys.executable, ...], env={"PYTHONPATH=os.pathsep.join(sys.path), **os.environ})

?

Bingo

Ok, so maybe an option to have the Pex runtime export PYTHONPATH as such before hand-off to use code. Please file away if that is a feature you want - it seems reasonable to me.

I labeled it as a feature request since I couldn't divine "don't want to" vs "can't" from Bingo. If it's can't and this blocks sane work over in Pants, some additional commentary on the issue would help prioritize. It certainly seems like there would be scenarios where it is a "can't" though even if yours is not one.

It's a "can but my users are throwing bricks and fits because 'this worked in Bazel'".

Ah system site-packages.

Even an interpreter found in a random venv will almost always have a site-packages with pip installed for example. So Pex scrubs the site-packages of any interpreter it discovers regardless of its provenance.

It's a "can but my users are throwing bricks and fits because 'this worked in Bazel'".

Ok, that counts as a can't in my book. I'll amend the issue.