to force pants to not use system python on OSX, we...
# general
f
to force pants to not use system python on OSX, we set
$PEX_PYTHON
appropriately. however, when running
./pants lint
, it seems like pants is still picking up system python. is there some other variable to set for that? or some better way of keeping pants away from system python?
e
See here:
Copy code
$ ./pants options --scope=python-setup
python-setup.artifact_cache_dir = None (from NONE)
python-setup.chroot_cache_dir = None (from NONE)
python-setup.interpreter_cache_dir = None (from NONE)
python-setup.interpreter_constraints = ['CPython>=2.7,<3'] (from HARDCODED)
python-setup.interpreter_search_paths = [] (from HARDCODED)
python-setup.platforms = ['current'] (from HARDCODED)
python-setup.resolver_allow_prereleases = None (from NONE)
python-setup.resolver_blacklist = {} (from HARDCODED)
python-setup.resolver_cache_dir = None (from NONE)
python-setup.resolver_cache_ttl = 315360000 (from HARDCODED)
python-setup.resolver_use_manylinux = True (from HARDCODED)
python-setup.setuptools_version = 33.1.1 (from HARDCODED)
python-setup.wheel_version = 0.29.0 (from HARDCODED)
You probably want to have:
Copy code
[python-setup]
interpreter_search_paths: [
    '/some/osx/path/brew/maybe?',
    '/some/ither/osx/path',
  ]
Unfortunately we don't natively support swithing this based on platform. If linux is CI-only, you can use
./pants --pants-config-files=pants.linux.ini ...
to override or nullify those paths.
đź‘Ť 1
f
can you just stack multiple paths that work for both linux and OSX, and as long as pants finds an interpreter somewhere there we’re good?
we have non-CI use of linux
e
Yes, that should work.
f
sigh… so judging from https://github.com/pantsbuild/pants/blob/60197188cb02f7ff60ac723f12780353fb2f2ee9/src/python/pants/backend/python/interpreter_cache.py#L100, it looks like pants is going to pick the minimum version it can find (and doesn’t, e.g., pick an interpreter from a path earlier in the list of search paths), and python on linux is at /usr/bin for us, which is where the bad system python is on OSX
so if we include /usr/bin, pants will use system python on OSX, and if we don’t, pants won’t find any python on Linux
e
Can you not also set
interpreter_constraints
in the same scope? Maybe
CPython>=2.7.11,<3
assuming you're on 2.7 and the wonky OSX python is 2.7.10.
f
so, we were actually doing this before on some old 1.3 beta, but then we upgraded to 1.7 and started running into these issues
I think someone looked into it and thought that maybe the patch version wasn’t actually being used for the constraint-checking? maybe this is fixed in 1.8 or something. or there’s some other reason this stopped working for us in 1.7
but so yeah FWIW, all of this is happening w/
interpreter_constraints: CPython>=2.7.11,<3
in our pants.ini under the
[python-setup]
section
e
And, for my understanding, the OSX system python should be excluded by that constraint but is not?
f
that’s my understanding. OSX system python is in fact 2.7.10
e
OK, Well interpreter-constraints works for us on master to avoid that very interpreter: https://github.com/pantsbuild/pants/blob/ad2223793c8c091d25eec5945ef9e3e0839d4a7b/.travis.yml#L261-L265 - the free variable is combining with interpreter-search-paths - which we do not use.
Perhaps your syntax for setting interpreter-constraints is different? Lists in pants options are subtle unfortunately. The link above shows list replacement, which is what you want. You maybe are appending, which is documented to be treated as OR.
You can check if you are appending by running
./pants options --scope=python-setup --name=interpreter-constraints
to see what pants thinks you're saying.
f
“You maybe are appending, which is documented to be treated as OR.” Ahhhh! We are appending. this must be what changed between our old 1.3 and 1.7
well, actually, it wasn’t clear we were appending, the line was:
Copy code
interpreter_constraints: CPython>=2.7.11,<3
which I guess is interpreted as an append when the base type is a list? maybe that var changed from a single string to a list
but the
./pants options
call showed that we were appending
e
Yes to all that. Great.