This is a follow-up to a (now lengthy) earlier thr...
# general
This is a follow-up to a (now lengthy) earlier thread. Can someone look over this mass of debug output to help me figure out how to get pants to build pylint.pex? Project: pants.toml: gist with debug output: env:
./pants lint :: --print-stacktrace -ldebug
./pants lint :: --no-process-execution-cleanup-local-dirs contents of
after I added
to the pip command output of running that modified I think this is the real error:
Copy code
pkg_resources.VersionConflict: (setuptools 44.0.0 (/Users/jafloyd/.cache/pants/named_caches/pex_root/pip.pex/cf4106b4c7898b462f7c172dde686a9747103f1a/.deps/setuptools), Requirement.parse('setuptools>=45')) Depending on the version of pants, the version of pylint, and a few settings, Pip complains it can’t find wrapt, or a few other deps, but that setuptools VersionConflict error always seems to be there. I’m not sure where to go from here.
👀 1
Thanks for the detailed report, this will be very helpful in debugging. Looking now.
Which platform were you getting these errors on? I can't reproduce this on MacOS, will try on Linux.
Never mind, I can see that this is MacOS
I'm on MacOS catalina using MacPorts installed python.
@proud-dentist-22844 what is the contents of
Yeah, this seems like the culprit: So that
is installed by setuptools (meaning you must have setuptools installed in
) and it sneaks in an importer that finds a version of distutils vendored by setuptools. If I read that correctly, it means setuptools sneaks itself onto the sys.path of your Python installation via that importer. So when Pex scrubs away
it does not manage to scrub away setuptools. That's my current guess anyhow. I'll try to set up the same scenario on my machine and see if I can get a repro...
👀 1
thanks for taking a look
That;s not it apparently. When I setup similar, Pex scrubs the .pth contamination
Copy code
$ PEX_VERBOSE=3 SETUPTOOLS_USE_DISTUTILS=local python3.6 pex -vvv "wrapt==1.12.1"
pex: Extracting pex to /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730: 154.3ms
pex: Executing unzipped pex for pex at /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730
pex: Installed VendorImporter(root='/home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730/.bootstrap', importables=(_Importable(module='attr', is_pkg=True, path='pex/vendor/_vendored/attrs', prefix='pex.third_party'), _Importable(module='pyparsing', is_pkg=False, path='pex/vendor/_vendored/packaging', prefix='pex.third_party'), _Importable(module='packaging', is_pkg=True, path='pex/vendor/_vendored/packaging', prefix='pex.third_party'), _Importable(module='setuptools', is_pkg=True, path='pex/vendor/_vendored/setuptools', prefix='pex.third_party'), _Importable(module='pkg_resources', is_pkg=True, path='pex/vendor/_vendored/setuptools', prefix='pex.third_party'), _Importable(module='wheel', is_pkg=True, path='pex/vendor/_vendored/wheel', prefix='pex.third_party')))
pex: Using $PATH constrained by ['>=2.7,<3.10,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*']
pex: Constraints on interpreters: ['>=2.7,<3.10,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*'], Matching Interpreter: /home/jsirois/.pyenv/versions/3.6.8/bin/python3.6
pex: Selecting runtime interpreter: 2.1ms
pex: Using the current interpreter /home/jsirois/.pyenv/versions/3.6.8/bin/python3.6 since it matches constraints and PYTHONPATH is not set.
pex: Found site-library: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Found .pth file: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages/distutils-precedence.pth
pex: Not a tainted path element: /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730/.bootstrap
pex: Not a tainted path element: /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/lib-dynload
pex: Tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Scrubbing from user site: /home/jsirois/.local/lib/python3.6/site-packages
pex: Scrubbing from site-packages: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Dropping _distutils_hack
pex: Dropping setuptools
contents of that pth file:
Copy code
import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'stdlib') == 'local'; enabled and __import__('_distutils_hack').add_shim();
hmm. I dont have “Dropping _distuils_hack” or “Dropping setuptools” in my debug output.
Yeah - you'd need to export SETUPTOOLS_USE_DISTUTILS=local to see that.
I do think this one is a dead end. That's the only bit that stood out in your output. I still can't repro.
Nor me
PEX_VERBOSE=3 SETUPTOOLS_USE_DISTUTILS=local python3.6 pex -vvv "wrapt==1.12.1"
end in this error for you?
Copy code
ModuleNotFoundError: No module named 'setuptools'
I just ran that command (well, with
) and it died with that error.
Yup. The scrubbing is very effective!
The only MacPorts patch that looks like it comes remotely close is That's on the good side of things though and removes items from site.
OK. So, I took the nuclear approach, and wiped out all installed macports apps. Then I reinstalled python 3.6, 3.7, 3.8, 3.9 + pip & virtualenv for each.
It successfully built the pylint.pex
So, something was messed up with my install.
Ok - I'm gald you're unstuck, but this is also the worst sort of problem. I'd love to understand what took place here.
Thanks for all the detailed information today.
👆 2
👍 1
It's great that you're able to make progress! But we (Pants) presumably have a problem still.
I used the mac os migrate utility to copy (amongst other things) my MacPorts install from one computer to another - that may have missed something somewhere.
Hopefully it was a wild goose chase and no one else will experience such ugly failures.
🤞 1