This is a follow-up to a (now lengthy) earlier thread. Can someone look over this mass of debug output to help me figure out how to get pants to build pylint.pex? Project: https://github.com/st2sandbox/st2/tree/pants pants.toml: https://github.com/st2sandbox/st2/blob/f0219d97e9e0d083cb11ea36eebe1348ed662110/pants.toml gist with debug output: https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd env: https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-0-pants-env

./pants lint :: --print-stacktrace -ldebug

https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-1-pants-lint-debug

./pants lint :: --no-process-execution-cleanup-local-dirs

https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-2-pants-no-process-execution-cleanup-local-dirs contents of

__run.sh

after I added

-vvv

to the pip command https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-3-__run-sh output of running that modified

__run.sh

https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-4-run-__run-sh I think this is the real error:

pkg_resources.VersionConflict: (setuptools 44.0.0 (/Users/jafloyd/.cache/pants/named_caches/pex_root/pip.pex/cf4106b4c7898b462f7c172dde686a9747103f1a/.deps/setuptools), Requirement.parse('setuptools>=45'))

https://gist.github.com/cognifloyd/a8cfc16c5d968865a09b34a9250c6fdd#file-4-run-__run-sh-L2857 Depending on the version of pants, the version of pylint, and a few settings, Pip complains it can’t find wrapt, or a few other deps, but that setuptools VersionConflict error always seems to be there. I’m not sure where to go from here.

Thanks for the detailed report, this will be very helpful in debugging. Looking now.

I'm on MacOS catalina using MacPorts installed python.

@proud-dentist-22844 what is the contents of

/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/distutils-precedence.pth

?

Yaaargh

That;s not it apparently. When I setup similar, Pex scrubs the .pth contamination

$ PEX_VERBOSE=3 SETUPTOOLS_USE_DISTUTILS=local python3.6 pex -vvv "wrapt==1.12.1"
pex: Extracting pex to /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730: 154.3ms
pex: Executing unzipped pex for pex at /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730
pex: Installed VendorImporter(root='/home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730/.bootstrap', importables=(_Importable(module='attr', is_pkg=True, path='pex/vendor/_vendored/attrs', prefix='pex.third_party'), _Importable(module='pyparsing', is_pkg=False, path='pex/vendor/_vendored/packaging', prefix='pex.third_party'), _Importable(module='packaging', is_pkg=True, path='pex/vendor/_vendored/packaging', prefix='pex.third_party'), _Importable(module='setuptools', is_pkg=True, path='pex/vendor/_vendored/setuptools', prefix='pex.third_party'), _Importable(module='pkg_resources', is_pkg=True, path='pex/vendor/_vendored/setuptools', prefix='pex.third_party'), _Importable(module='wheel', is_pkg=True, path='pex/vendor/_vendored/wheel', prefix='pex.third_party')))
pex: Using $PATH constrained by ['>=2.7,<3.10,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*']
pex: Constraints on interpreters: ['>=2.7,<3.10,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*'], Matching Interpreter: /home/jsirois/.pyenv/versions/3.6.8/bin/python3.6
pex: Selecting runtime interpreter: 2.1ms
pex: Using the current interpreter /home/jsirois/.pyenv/versions/3.6.8/bin/python3.6 since it matches constraints and PYTHONPATH is not set.
pex: Found site-library: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Found .pth file: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages/distutils-precedence.pth
pex: Not a tainted path element: /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730/.bootstrap
pex: Not a tainted path element: /home/jsirois/.pex/unzipped_pexes/fe5a6e33c47ab8f54825dc7dd841561189270730
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python36.zip
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6
pex: Not a tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/lib-dynload
pex: Tainted path element: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Scrubbing from user site: /home/jsirois/.local/lib/python3.6/site-packages
pex: Scrubbing from site-packages: /home/jsirois/.pyenv/versions/3.6.8/lib/python3.6/site-packages
pex: Dropping _distutils_hack
pex: Dropping setuptools
...

contents of that pth file:

import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'stdlib') == 'local'; enabled and __import__('_distutils_hack').add_shim();

Yeah - you'd need to export SETUPTOOLS_USE_DISTUTILS=local to see that.

Nor me

Does

PEX_VERBOSE=3 SETUPTOOLS_USE_DISTUTILS=local python3.6 pex -vvv "wrapt==1.12.1"

end in this error for you?

ModuleNotFoundError: No module named 'setuptools'

Yup. The scrubbing is very effective!

OK. So, I took the nuclear approach, and wiped out all installed macports apps. Then I reinstalled python 3.6, 3.7, 3.8, 3.9 + pip & virtualenv for each.

Hrm.

It's great that you're able to make progress! But we (Pants) presumably have a problem still.

I used the mac os migrate utility to copy (amongst other things) my MacPorts install from one computer to another - that may have missed something somewhere.