https://pantsbuild.org/ logo
#general
Title
# general
f

fierce-park-88503

08/23/2018, 10:53 PM
to force pants to not use system python on OSX, we set
$PEX_PYTHON
appropriately. however, when running
./pants lint
, it seems like pants is still picking up system python. is there some other variable to set for that? or some better way of keeping pants away from system python?
e

enough-analyst-54434

08/24/2018, 1:16 AM
See here:
Copy code
$ ./pants options --scope=python-setup
python-setup.artifact_cache_dir = None (from NONE)
python-setup.chroot_cache_dir = None (from NONE)
python-setup.interpreter_cache_dir = None (from NONE)
python-setup.interpreter_constraints = ['CPython>=2.7,<3'] (from HARDCODED)
python-setup.interpreter_search_paths = [] (from HARDCODED)
python-setup.platforms = ['current'] (from HARDCODED)
python-setup.resolver_allow_prereleases = None (from NONE)
python-setup.resolver_blacklist = {} (from HARDCODED)
python-setup.resolver_cache_dir = None (from NONE)
python-setup.resolver_cache_ttl = 315360000 (from HARDCODED)
python-setup.resolver_use_manylinux = True (from HARDCODED)
python-setup.setuptools_version = 33.1.1 (from HARDCODED)
python-setup.wheel_version = 0.29.0 (from HARDCODED)
You probably want to have:
Copy code
[python-setup]
interpreter_search_paths: [
    '/some/osx/path/brew/maybe?',
    '/some/ither/osx/path',
  ]
Unfortunately we don't natively support swithing this based on platform. If linux is CI-only, you can use
./pants --pants-config-files=pants.linux.ini ...
to override or nullify those paths.
👍 1
f

fierce-park-88503

08/24/2018, 1:31 PM
can you just stack multiple paths that work for both linux and OSX, and as long as pants finds an interpreter somewhere there we’re good?
we have non-CI use of linux
e

enough-analyst-54434

08/24/2018, 1:34 PM
Yes, that should work.
f

fierce-park-88503

08/24/2018, 2:45 PM
sigh… so judging from https://github.com/pantsbuild/pants/blob/60197188cb02f7ff60ac723f12780353fb2f2ee9/src/python/pants/backend/python/interpreter_cache.py#L100, it looks like pants is going to pick the minimum version it can find (and doesn’t, e.g., pick an interpreter from a path earlier in the list of search paths), and python on linux is at /usr/bin for us, which is where the bad system python is on OSX
so if we include /usr/bin, pants will use system python on OSX, and if we don’t, pants won’t find any python on Linux
e

enough-analyst-54434

08/24/2018, 3:08 PM
Can you not also set
interpreter_constraints
in the same scope? Maybe
CPython>=2.7.11,<3
assuming you're on 2.7 and the wonky OSX python is 2.7.10.
f

fierce-park-88503

08/24/2018, 3:15 PM
so, we were actually doing this before on some old 1.3 beta, but then we upgraded to 1.7 and started running into these issues
I think someone looked into it and thought that maybe the patch version wasn’t actually being used for the constraint-checking? maybe this is fixed in 1.8 or something. or there’s some other reason this stopped working for us in 1.7
but so yeah FWIW, all of this is happening w/
interpreter_constraints: CPython>=2.7.11,<3
in our pants.ini under the
[python-setup]
section
e

enough-analyst-54434

08/24/2018, 3:21 PM
And, for my understanding, the OSX system python should be excluded by that constraint but is not?
f

fierce-park-88503

08/24/2018, 3:22 PM
that’s my understanding. OSX system python is in fact 2.7.10
e

enough-analyst-54434

08/24/2018, 3:38 PM
OK, Well interpreter-constraints works for us on master to avoid that very interpreter: https://github.com/pantsbuild/pants/blob/ad2223793c8c091d25eec5945ef9e3e0839d4a7b/.travis.yml#L261-L265 - the free variable is combining with interpreter-search-paths - which we do not use.
Perhaps your syntax for setting interpreter-constraints is different? Lists in pants options are subtle unfortunately. The link above shows list replacement, which is what you want. You maybe are appending, which is documented to be treated as OR.
You can check if you are appending by running
./pants options --scope=python-setup --name=interpreter-constraints
to see what pants thinks you're saying.
f

fierce-park-88503

08/24/2018, 3:42 PM
“You maybe are appending, which is documented to be treated as OR.” Ahhhh! We are appending. this must be what changed between our old 1.3 and 1.7
well, actually, it wasn’t clear we were appending, the line was:
Copy code
interpreter_constraints: CPython>=2.7.11,<3
which I guess is interpreted as an append when the base type is a list? maybe that var changed from a single string to a list
but the
./pants options
call showed that we were appending
e

enough-analyst-54434

08/24/2018, 4:03 PM
Yes to all that. Great.
5 Views