https://pantsbuild.org/ logo
#general
Title
# general
f

flaky-artist-57016

08/01/2022, 9:03 PM
Hello, I’m trying to determine which Python installation
pants
is using when I call
./pants test ::
for example. When I run the test command with
-ldebug
it appears (I could be wrong) that
pants
is spawning multiple processes with different versions of Python that I have installed--even if I remove them from my
PATH
. For example:
Copy code
16:42:45.42 [DEBUG] spawned local process as Some(23549) for Process { argv: ["/Users/username/miniconda3_py3.8/envs/io-env/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23550) for Process { argv: ["/Users/username/.pyenv/versions/3.8.13/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23551) for Process { argv: ["/Users/username/.pyenv/versions/3.10.4/bin/python"...
...
What is going on there?
h

happy-kitchen-89482

08/01/2022, 9:41 PM
Hi, sorry for the trouble. The way this works is: The
python-bootstrap
options control where Python looks for possible interpreters. The
python
options, along with the
interpreter_constraints
fields, if any, on your targets, control which of those interpreters is actually selected for a given process.
Pants will ensure that an interpreter that is compatible with the constraints of the target set (which defaults to the constraints configured for the entire repo).
So, first question is, what is in your pants.toml under
[python-bootstrap]
and
[python]
sections?
f

flaky-artist-57016

08/01/2022, 9:43 PM
interpreter_constraints = [">=3.8,<3.9"]
search_path = ["<PATH>", "<PYENV>"]
h

happy-kitchen-89482

08/01/2022, 9:50 PM
And is
miniconda
mentioned in any constraints fields of any targets?
">=3.8,<3.9"
is shorthand for
"CPython>=3.8,<3.9"
, so i'm surprised miniconda is being selected
But a target-specific value would override that
f

flaky-artist-57016

08/01/2022, 9:57 PM
Gotcha. I believe I had set
$PYTHON
to the Python executable in my miniconda env when looking into this. Before explicitly setting
$PYTHON
I was expecting that pants would pick up the miniconda Python install because it is quite early first in my
PATH
. I’ve since used pyenv to install the same version and pants is picking that up instead.
To clarify on my last statement, I have rerun
./pants -ldebug test ::
after using pyenv to install the same version of Python as in my conda environment and pants is exclusively using that. I’m not seeing different versions being used in different processes. Seems like
pants
just can’t detect Python installed in a conda env unless the path is explicitly set in
$PYTHON
?
h

happy-kitchen-89482

08/01/2022, 10:49 PM
I guess not, possibly the logic Pants uses to detect that looks like a python interpreter is in fact one (and to extract its version) doesn't work on miniconda
How did you install miniconda?
f

flaky-artist-57016

08/01/2022, 11:04 PM
I used the bash installer script for Intel Macs available here: https://docs.conda.io/en/latest/miniconda.html
h

happy-kitchen-89482

08/02/2022, 12:36 AM
When you say you used pyenv to install the same version, do you mean to install miniconda? Or cpython at the same python version?
f

flaky-artist-57016

08/02/2022, 12:37 AM
Vanilla python at the same version.
e.g.,
pyenv install 3.8.13
h

happy-kitchen-89482

08/02/2022, 12:38 AM
Right, so it makes sense those would get picked up, since they are compatible with your interpreter constraints
but miniconda is not, so I would not expect it to be used
f

flaky-artist-57016

08/02/2022, 12:54 AM
Thanks for the explanation. Just to make sure I understand: Pants is strictly looking for a CPython interpreter that fits the constraints, but the interpreter that miniconda installs “looks” different even if it is at the same Python version.
h

happy-kitchen-89482

08/02/2022, 1:28 AM
I assume so. When you run this on miniconda what do you get?
import sys; print(sys.implementation)
The interpreter selection process takes the implementation into account
>=3.8,<3.9
is shorthand for
CPython>=3.8,<3.9
so that's really what your interpreter constraints are asking for
f

flaky-artist-57016

08/02/2022, 1:29 AM
namespace(_multiarch='darwin', cache_tag='cpython-38', hexversion=50859504, name='cpython', version=sys.version_info(major=3, minor=8, micro=13, releaselevel='final', serial=0))
h

happy-kitchen-89482

08/02/2022, 1:29 AM
huh
interesting
in that case, miniconda advertises itself as cpython I guess?
I know nothing about miniconda so I am shooting in the dark here
From my quick googling it looks like conda is just cpython with some extra libraries prepackaged
f

flaky-artist-57016

08/02/2022, 1:32 AM
Gotcha. It’s just a “mini” version of the Anaconda distribution that is popular in the sciences.
Yep
h

happy-kitchen-89482

08/02/2022, 1:33 AM
So in that case it would be picked up by Pants interpreter selection
Which is what you saw earlier, but I am confused as to why it didn't use it consistently
And now you're saying that it prefers the pyenv-installed one?
f

flaky-artist-57016

08/02/2022, 1:44 AM
I’ve been messing around with it for some time this evening. I cleared the pants caches and started a new session and pants is now picking up the miniconda interpreter and only that interpreter. I must have polluted my path or otherwise confused pants at some point.
I really appreciate your assistance. I’ve learned a bit more about pants.
h

happy-kitchen-89482

08/02/2022, 2:16 AM
you're welcome. I'm still finding it odd that Pants picked up different interpreters for the same constraints in a single run
If you see that happen again please let us know!
👍 1