#2235 `secrets` not in `sys.path` when using `--venv` Issue created by yjabri Overview This arose from a discussion on slack. The repo https://github.com/yjabri/debug-pex-import describes how to reproduce the bug in the

README.md

. The culprit appears to be pex_bootstrapper.py#L672. Just prior to the line, in a debugger I can successfully

import secrets

. (A cause of

flask

failing to import). However just after the line, in a new debugger session

import secrets

raises a

ModuleNotFoundError

exception. Some details The line <pex_bootstrapper.py#L672|https://github.com/pantsbuild/pex/blob/main/pex/pex_bootstrapper.py#L672> converts my

sys.path

from

['/home/yjabri/Projects/debug-pex-import',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python310.zip',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10/lib-dynload',
'',
'/home/yjabri/.local/lib/python3.10/site-packages',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10/site-packages',
'dist/.pex' ]

to

['/home/yjabri/.pex/venvs/6c8eaec5b07b6fe5d8c257cd72d1297dfafdfc8e/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.10/site-packages',
'dist/.pex']

My understanding is that this doesn't include standard library which would be in

'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10'

For reference,

python dist/.pex -c "import sys; print(sys.path)"

outputs

['',
'/home/yjabri/.pex/venvs/6c8eaec5b07b6fe5d8c257cd72d1297dfafdfc8e/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python310.zip',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10',
'/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10/lib-dynload',
'/home/yjabri/.pex/venvs/6c8eaec5b07b6fe5d8c257cd72d1297dfafdfc8e/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.10/site-packages']

Why does this happen? If I set

sys.path = [ 
    '/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10',
    '/home/yjabri/.pex/venvs/6c8eaec5b07b6fe5d8c257cd72d1297dfafdfc8e/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.10/site-packages',
    'dist/.pex'
]

I can

import secrets

without a

ModuleNotFoundError

. In general, if I create a

venv

manually via

python -m venv venv

and run

venv/bin/python -c "import sys; print(sys.path)"

I see

['', '/home/yjabri/.pyenv/versions/3.10.13/lib/python310.zip', '/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10', '/home/yjabri/.pyenv/versions/3.10.13/lib/python3.10/lib-dynload', '/tmp/foobar/venv/lib/python3.10/site-packages']

I get

ModuleNotFoundError

by running

venv/bin/python -c "import sys; sys.path = sys.path[3:]; import secrets"

but not

venv/bin/python -c "import sys; sys.path = sys.path[2:]; import secrets"

does not raise the exception. If I replace

secrets

with other standard library packages like

http

or

random

I get the same

ModuleNotFoundError

exception. pantsbuild/pex