https://pantsbuild.org/ logo
Join Slack
Powered by
Hello, I’m trying to determine which Python instal...
# general
f
Hello, I’m trying to determine which Python installation 
pants
is using when I call 
./pants test ::
for example. When I run the test command with 
-ldebug
it appears (I could be wrong) that 
pants
is spawning multiple processes with different versions of Python that I have installed--even if I remove them from my 
PATH
. For example:
Copy code
16:42:45.42 [DEBUG] spawned local process as Some(23549) for Process { argv: ["/Users/username/miniconda3_py3.8/envs/io-env/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23550) for Process { argv: ["/Users/username/.pyenv/versions/3.8.13/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23551) for Process { argv: ["/Users/username/.pyenv/versions/3.10.4/bin/python"...
...
What is going on there?
h
Hi, sorry for the trouble. The way this works is: The 
python-bootstrap
options control where Python looks for possible interpreters. The 
python
options, along with the 
interpreter_constraints
fields, if any, on your targets, control which of those interpreters is actually selected for a given process.
Pants will ensure that an interpreter that is compatible with the constraints of the target set (which defaults to the constraints configured for the entire repo).
So, first question is, what is in your pants.toml under 
[python-bootstrap]
and 
[python]
sections?
f
interpreter_constraints = [">=3.8,<3.9"]
search_path = ["<PATH>", "<PYENV>"]
h
And is 
miniconda
mentioned in any constraints fields of any targets?
">=3.8,<3.9"
is shorthand for 
"CPython>=3.8,<3.9"
, so i'm surprised miniconda is being selected
But a target-specific value would override that
f
Gotcha. I believe I had set 
$PYTHON
to the Python executable in my miniconda env when looking into this. Before explicitly setting 
$PYTHON
I was expecting that pants would pick up the miniconda Python install because it is quite early first in my 
PATH
. I’ve since used pyenv to install the same version and pants is picking that up instead.
To clarify on my last statement, I have rerun 
./pants -ldebug test ::
after using pyenv to install the same version of Python as in my conda environment and pants is exclusively using that. I’m not seeing different versions being used in different processes. Seems like 
pants
just can’t detect Python installed in a conda env unless the path is explicitly set in 
$PYTHON
?
h
I guess not, possibly the logic Pants uses to detect that looks like a python interpreter is in fact one (and to extract its version) doesn't work on miniconda
How did you install miniconda?
f
I used the bash installer script for Intel Macs available here: https://docs.conda.io/en/latest/miniconda.html
h
When you say you used pyenv to install the same version, do you mean to install miniconda? Or cpython at the same python version?
f
Vanilla python at the same version.
e.g., 
pyenv install 3.8.13
h
Right, so it makes sense those would get picked up, since they are compatible with your interpreter constraints
but miniconda is not, so I would not expect it to be used
f
Thanks for the explanation. Just to make sure I understand: Pants is strictly looking for a CPython interpreter that fits the constraints, but the interpreter that miniconda installs “looks” different even if it is at the same Python version.
h
I assume so. When you run this on miniconda what do you get? 
import sys; print(sys.implementation)
The interpreter selection process takes the implementation into account
>=3.8,<3.9
is shorthand for 
CPython>=3.8,<3.9
so that's really what your interpreter constraints are asking for
f
namespace(_multiarch='darwin', cache_tag='cpython-38', hexversion=50859504, name='cpython', version=sys.version_info(major=3, minor=8, micro=13, releaselevel='final', serial=0))
h
huh
interesting
in that case, miniconda advertises itself as cpython I guess?
I know nothing about miniconda so I am shooting in the dark here
From my quick googling it looks like conda is just cpython with some extra libraries prepackaged
f
Gotcha. It’s just a “mini” version of the Anaconda distribution that is popular in the sciences.
Yep
h
So in that case it would be picked up by Pants interpreter selection
Which is what you saw earlier, but I am confused as to why it didn't use it consistently
And now you're saying that it prefers the pyenv-installed one?
f
I’ve been messing around with it for some time this evening. I cleared the pants caches and started a new session and pants is now picking up the miniconda interpreter and only that interpreter. I must have polluted my path or otherwise confused pants at some point.
I really appreciate your assistance. I’ve learned a bit more about pants.
h
you're welcome. I'm still finding it odd that Pants picked up different interpreters for the same constraints in a single run
If you see that happen again please let us know!
👍 1
Open in Slack
PreviousNext
Powered by