https://pantsbuild.org/ logo
Join Slack
Powered by
Hi :slightly_smiling_face: I'm just taking a first...
# general
r
Hi 🙂 I'm just taking a first look at Pants and seem to have run into problems early on, I've had a look but can't find any info suggesting where to look 😞 I've made a very simple repo with a couple of projects, using poetry with a pyproject.toml in the root If I add in a dependency (poetry add x) that's working fine, adding into the pyproject.toml and poetry.lock as expected For some reason if I add in pandas / numpy as an import in one of the src files, it fails. Other dependencies (to pick some random ones, pytz and requests) are both fine. It's coming back with:
Exception message: 1 Exception encountered:
ProcessExecutionFailure: Process 'Building requirements.pex with 3 requirements: pandas<2.0.0,>=1.3.3, pytz<2022.0.0,>=2021.1, requests<3.0.0,>=2.26.0' failed with exit code 1.
stdout:
stderr:
ERROR: Could not find a version that satisfies the requirement pandas<2.0.0,>=1.3.3
ERROR: No matching distribution found for pandas<2.0.0,>=1.3.3
pandas 1.3.3 definitely exists, I'm not sure why specifically pandas and numpy are causing problems but doing the same thing for other dependencies is fine 🤔 Any suggestions on where to look for this? Thanks!
h
Hi! Which python version are you targeting? And does that pandas version support it? It's probably not that, but worth checking.
b
Could be an architecture issue? Those are both deps with native code, right?
r
🤔 my pyproject.toml has
python = ">=3.8,<3.11"
the venv in pycharm has spun up 3.9.5 ..but weirdly:
============================= test session starts ==============================
platform linux -- Python 3.6.14, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
it looks like pants is trying to use 3.6? I am not sure where that could have even come from 🤔
hm, adding in:
[python-setup]
interpreter_constraints = ["CPython==3.8.*"]
is complaining about things to do with 3.6 some more, I'll see if I can find where that could be coming from 🤔
(make that 3.9 rather than 3.8, since I don't have 3.8 on here)
Actually, with 3.8 that's possibly more helpful in that it is erroring with some info, specifying 3.9 is working, I'm not quite sure how I missed the 3.6 in the error messages, thanks for the nudge in that direction again! It looks like it's picking up an unexpected version of python for some reason, I'm not sure if that's on the Pants side or configured elsewhere?
stderr:
Could not find a compatible interpreter.
Examined the following working interpreters:
1.) /home/mgeeves/.pyenv/versions/3.6.14/bin/python3.6 CPython==3.6.14
2.) /usr/bin/python3.9 CPython==3.9.5
Skipped the following broken interpreters:
1.) /home/mgeeves/.pyenv/shims/python3.6:
pyenv: python3.6: command not found
The `python3.6' command exists in these Python versions:
3.6.14
I would have expected Pants to create the venv using the "system default" (or the venv currently in use even) - in this case 3.9.5 for both - over the pyenv one but maybe that's intentional? Let me know if that sounds odd and I can provide any info, otherwise I'll just remember to always specify the interpreter_constraints version and at some point take another look at pyenv 🤔
I think that does look not-quite-right on the pants side:
mgeeves@bearly:~$ pyenv versions
* system (set by /home/mgeeves/.pyenv/version)
3.6.14
mgeeves@bearly:~$ pyenv global
system
mgeeves@bearly:~$ python --version
Python 3.9.5
mgeeves@bearly:~$ pyenv install 3.9.5
Downloading Python-3.9.5.tar.xz...
-> https://www.python.org/ftp/python/3.9.5/Python-3.9.5.tar.xz
Installing Python-3.9.5...
Installed Python-3.9.5 to /home/mgeeves/.pyenv/versions/3.9.5
mgeeves@bearly:~$ pyenv versions
* system (set by /home/mgeeves/.pyenv/version)
3.6.14
3.9.5
mgeeves@bearly:~$ pyenv global 3.9.5
mgeeves@bearly:~$ pyenv versions
system
3.6.14
* 3.9.5 (set by /home/mgeeves/.pyenv/version)
As far as pyenv is concerned, 3.6.14 isn't expected to be the one in use, however even with a .python-version file as well as the global being set to 3.9.5, if I don't specify interpreter_constraints I get:
pid 4150743 -> /home/mgeeves/.cache/pants/named_caches/pex_root/venvs/202fc486cd05d66fe2f93e6bcb0f03fb4a9fe241/83a29cf2dfb126e8fb81d2f6e6b8859e39535f04/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/mgeeves/.cache/pants/named_caches/pex_root --log /tmp/process-executionDzVB86/.tmp/tmpikiew1xj/pip.log download --dest /tmp/process-executionDzVB86/.tmp/tmpynwn177c/home.mgeeves..pyenv.versions.3.6.14.bin.python3.6 numpy<2.0.0,>=1.21.2 pandas<2.0.0,>=1.3.3 pytz<2022.0.0,>=2021.1 requests<3.0.0,>=2.26.0 --index-url https://pypi.org/simple/ --retries 5 --timeout 15
I don't think it should be picking up 3.6 there 🤔 (again, fine for me now I know what to add, let me know if there's anything else useful I could look at etc)
I will stop typing and go to sleep any time now.. 😄 I see in python_setup.py, if using <PYENV_LOCAL> then it will return a list with the single entry found in .python-version <PYENV> will return the sorted list of all versions available on the system, and then later it looks like the first entry is used. Is that intentional? If a ${PYENV_ROOT}/version exists that would be a better candidate? 🤔 Also, alphabetically sorted order probably isn't what you would expect, I see from a quick google there are simple distutils imports to handle sorting, happy to push up a PR if it would be of use/interest, or also happy to move on if it's working as intended. Off to sleep now! 🙂
h
I always find interpreter selection issues confusing, but possibly @enough-analyst-54434 can shed light here
But basically, Pants tries to find a suitable interpreter based on your configured interpreter constraints, rather than just use whatever is first on your PATH.
So that it can control the interpreter version (this is especially necessary if different parts of your repo require different interpreter versions).
And as you discovered, you configure the interpreter version constraints with [this option](https://www.pantsbuild.org/docs/reference-python-setup#section-interpreter-constraints)
Then you use [this option](https://www.pantsbuild.org/docs/reference-python-setup#section-interpreter-search-paths) to control where Pants searches for possible interpreters
Which you've also discovered (I just fixed a rendering bug on that page BTW, so now the special strings actually display properly)
But to clarify, that is how Pants discovers the set of actual interpreters from which to select one based on the interpreter constraints.
So for example setting the option to 
["<PYENV>"]
means "the only interpreters Pants will use are pyenv interpreters, never mind what is on the PATH. But then to decide which of those pyenv interpreters to use, you set 
interpreter_constraints
Does that all make sense?
We should probably document this better
r
Yes, it makes sense, thanks for explaining further 🙂
based on your configured interpreter constraints
My problem was as a result of not configuring it, and the behaviour of the undefined default not being what I expected.
I didn't come across "needing to" configure in the Getting Started, it might be worth mentioning somewhere, basic setup to include it or in the list of issues....but equally my use case of having pyenv installed but not actively trying to use specific versions from the start may be a rare one 😄
h
Thanks for the feedback - if you've encountered this issue, probably others will too.
👍 1
2 Views
Open in Slack
PreviousNext
Powered by