Weird thing happening, I hope someone will be able to explain, I was about to get crazy trying to figure it out. Suddenly running one of my apps (a PEX binary) started to give errors about requests not finding module chardet. The message came from /home/me/.local/lib/python-3.8, but the PEX had requests 2.31. Removing the local lib helped, but • why does pex even look at the local python environment? • why does the local env have priority? If some library had deps on 2.22, I guess pants would have given an error message since it packs 2.31 • how can I avoid such things in the future? I had not installed anything new recently, I guess the 2.22 version has been there for quite a while Thanks in advance!

PEX files studiously avoid using the local environment; so your questions can stop at bullet 1 - this is a bug, unless you built the PEX with

--inherit-path={prefer,fallback}

or run it in an environment that exports

PEX_INHERIT_PATH={prefer,fallback}

.

ok, I'll try that

That will not fix, it will just show information from Pex about selecting and rejecting various sys.path elements.

Yes. Unfortunately, reinstalling requests in the local env doesn't seem to reactivate the problem.

Yeah - this is critical data. There are no known issues with environment isolation for PEX files; so I don't have a clue here. There is 1 known issue with using the magic

__pex__

import hook, which I just fixed but is not released. Presumably this is a PEX file you run though and not a PEX file you import from using the magic

__pex__

import hook.

Just a plain pex, no magic anything.

Even having saved the actual command line stack trace would have maybe been helpful; so try to remember not to throw out console session data if frustration is creeping in!

I have the traceback. Truncated the start, as it's all inside the pex env.

File "/home/vdumitre/.pex/unzipped_pexes/d65861d321733e128b2971be3982baf406a579fa/scoop/shell.py", line 28, in <module>
    from scoop.vault import logger_deny_list
  File "/home/vdumitre/.pex/unzipped_pexes/d65861d321733e128b2971be3982baf406a579fa/scoop/vault.py", line 6, in <module>
    import hvac
  File "/home/vdumitre/.pex/installed_wheels/622e9eba30d74ff6253d380cd04e5253ceccd75aa3d59336988b691b256712ea/hvac-0.10.5-py2.py3-none-any.whl/hvac/__init__.py", line 1, in <module>
    from hvac.v1 import Client
  File "/home/vdumitre/.pex/installed_wheels/622e9eba30d74ff6253d380cd04e5253ceccd75aa3d59336988b691b256712ea/hvac-0.10.5-py2.py3-none-any.whl/hvac/v1/__init__.py", line 7, in <module>
    from hvac import aws_utils, exceptions, adapters, utils, api
  File "/home/vdumitre/.pex/installed_wheels/622e9eba30d74ff6253d380cd04e5253ceccd75aa3d59336988b691b256712ea/hvac-0.10.5-py2.py3-none-any.whl/hvac/aws_utils.py", line 4, in <module>
    import requests
  File "/home/vdumitre/.local/lib/python3.8/site-packages/requests-2.22.0-py3.8.egg/requests/__init__.py", line 44, in <module>
    import chardet
ModuleNotFoundError: No module named 'chardet'

Ok, great. Yeah - that is not enough in this case. The PEX_VERBOSE logs are needed. But, generally, you should lead with this sort of detailed data! Always good.

I thought I was doing something wrong 🙂

One other useful bit would be how do you run the PEX?

./my.pex

or

/this/python my.pex

? And how do you get things installed in

~/.local/lib/python3.8

and what OS exactly? Fedora X or Ubuntu Y or?

I run it directly ./my.pex, The local env is just installed with "pip install" for a few tools that I need globally. It is Ubuntu 20.04.

So you're claiming

pip install

with no special flags installs to

~/.local/...

? Is this system

pip

or

pip

from an activated venv or ...?

system pip (~/.local/bin/pip), and without "--user".

Ok, ~/.local anything is not system. So you have done some setup above and beyond stock Ubuntu 20.04 + stock Ubuntu 20.04 Python 3.8

I am trying to remember... Can it be that ~/.local is where "--user" installations go? Maybe I did that in the beginning (or one of the tools I'm using did). Usually I keep things in venvs, to avoid this kind of issues, but there are tools that I didn't know how to install otherwise than the way their documentation suggested.

Yes

--user

installations go to

~/.local/

.