https://pantsbuild.org/ logo
Join Slack
Powered by
Hello! My system python version was updated to 3.9...
# general
s
Hello! My system python version was updated to 3.9.x (because of Ubuntu 20.04 -> 21.04 upgrade) Now my IDEA complains about python version when I try to refresh the project:
Copy code
No valid Python interpreter found. For `pants_version = "1.30.0"`, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.
If I open IDEA's terminal and run 
python
there, I will see:
Copy code
% python
Python 3.8.6 (default, May  4 2021, 09:31:45) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Cause I have pyenv configured for this (scala) project So I don't get how to get Pants in IDEA up and running cause it seems like it tries to use 3.9.x and fails (and it ignored pyenv) Any ideas how to fix it? Cheers.
h
Hi! Is this an issue with the Pants IDEA plugin?
It sounds like you need a 3.8 interpreter to be on IDEA's own PATH
how to set that is system-dependent, not sure how to do so for Ubuntu
Your experiment with IDEA's terminal might mean that that interpreter is already on the PATH, but maybe not?
Can you 
echo $PATH
in that terminal?
s
Hi! That's my $PATH in IDEA's terminal:
Copy code
% echo $PATH
/home/serge/bin:/home/serge/.nvm/versions/node/v14.16.1/bin:/home/serge/.pyenv/plugins/pyenv-virtualenv/shims:/home/serge/.pyenv/shims:/home/serge/.pyenv/bin:/home/serge/.cargo/bin:/home/serge/.local/bin:/home/serge/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
h
Thanks! So it looks like 
python
in IntelliJ will most likely come from 
/home/serge/.pyenv/shims
, which I suspect means it'll use the pyenv global interpreter (since IntelliJ is probably not running in a shell or a CWD in which you've set some other pyenv interpreter)
Which interpreter does 
pyenv global
show?
If that is 3.9 then as a temporary workaround you could change the pyenv global interpreter to 3.8, until we figure out how the plugin selects an interpreter to use.
@wide-energy-11069 any ideas on this? (pun intended)
s
Copy code
~ % pyenv global
3.8.6
the same in IDEA
h
Hrm
So it's now down to how the v1 intellij plugin locates its interpreter, which apparently is not via the regular IntelliJ PATH
@wide-energy-11069 is the expert here
w
Now my IDEA complains about python version when I try to refresh the project:
No valid Python interpreter found. For 
pants_version = “1.30.0”
, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.
this should be related to how pants find the interpreter. is py36,37,38 on your system path?
Copy code
which -a python3
if not, make them available, and retry after restarting intellij. if still no dice and your pants is a pex, you can use /etc/pexrc to help it find it. E.g.
Copy code
$ cat /etc/pexrc
PEX_PYTHON_PATH=/opt/ee/python/2.7/bin/python2.7:/<some/path>/3.6/bin/python3.6
s
checked out python3 locations:
Copy code
~ % which -a python3
/home/serge/.pyenv/shims/python3
/usr/bin/python3
/bin/python3
/home/serge/.pyenv/shims/python3
is 3.8.6 other 2 are 3.9.4
Copy code
% which -a python                 
/home/serge/.pyenv/shims/python
this one is also 3.8.6
w
iirc intellij won’t respect venv when running pants, so the right python needs to be on system path.
s
I assume that changing 
/bin/python3
and 
/usr/bin/python3
from 3.9.x to 3.8.x can lead to some unforeseen problems with the operating system itself, cause all the packages will assume 3.9.x and not 3.8.x
w
so have you tried the /etc/pexrc approach?
s
I really don't understand how to check if "my pants is a pex"
h
What happens if you put 
PEX_PYTHON_PATH=/path/to/python3.8
in 
/etc/pexrc
?
Oh, and it may be that 1.30.x doesn't handle pyenv shims well. You could try adding the path to the real underlying interpreter (e.g., 
/Users/<username>/.pyenv/versions/3.8.6/bin/python
) to intellij's PATH and restart intellij.
I vaguely remember 1.30 having issues with pyenv shims
s
but I don't have 
/etc/pexrc
h
Can you create it?
It's probably not that, but can't hurt to rule it out.
s
I created it
Copy code
~ % cat /etc/pexrc 
PEX_PYTHON_PATH=/home/serge/.pyenv/versions/3.8.6/bin/python3.8
but it did not help
h
Right, so I think the important thing is to put the "real" path to the 3.8 interpreter (not the shim) on IntelliJ's PATH
I am not sure how to set an env var for a GUI app in MacOS though
but I assume there is a way
@wide-energy-11069 What sets the Intellij binary's PATH?
w
it just takes the system PATH + ~/.bashXXX loaded for terminal
PEX_PYTHON_PATH=/home/serge/.pyenv/versions/3.8.6/bin/python3.8
in case this doesn’t work. this might have to be path to real py38 interpreter, not in venv
also running pex with a venv probably isn’t a good idea - if there are already libraries installed in venv, it may conflict with whatever the pex brings in.
h
I don't think pex is relevant here?
This is the actual PATH we need to affect?
w
/etc/pexrc
is relevant for pex (assuming the pants run here is in a pex), especially if the right interpreter for pex isn’t available on system PATH.
h
I don't think the Pants run here is a pex? I assume it's installed via a runner script from the wheels
w
oh ok. in case that please ignore pexrc
h
How does one set env vars on GUI-launched applications like IntelliJ on OSX?
w
ah yeah, it’s available in the run configurations.
so the downside is that this needs to be configured manually every time, or requires a custom plugin to overwrite every time
s
but how do I set an env variable for Pants IDEA plugin? I cannot even import a project.
w
@happy-kitchen-89482 I guess there could be a larger question in general. If Pants were to run where the right interpreter isn’t on the system PATH, should there be a way to tell Pants how to find it, similar to /etc/pexrc for pex. Could be a separate env var that only PANTS will use e.g. 
PANTS_INTERPRETOR_PATH
. just throwing this idea out.
h
There are ways for pants in general, what I'm not clear on is how IntelliJ invokes pants
does it call the 
./pants
wrapper script in the repo?
because it just occurs to me that you can do whatever PATH manipulations you want in that script...
w
yes, it calls 
./pants
exactly, and that’s how twitter does it. so 
./pants
is essentially a wrapper which eventually calls the pants.pex in our case
s
I tried tinkering with IDEA Absolute Path Variables restarting it every time added the whole PATH there - no effect
Copy code
/home/serge/bin:/home/serge/.nvm/versions/node/v14.16.1/bin:/home/serge/.pyenv/plugins/pyenv-virtualenv/shims:/home/serge/.pyenv/shims:/home/serge/.pyenv/bin:/home/serge/.cargo/bin:/home/serge/.local/bin:/home/serge/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
added only 
PATH=/home/serge/.pyenv/versions/3.8.6/bin
- no effect added only 
PEX_PYTHON_PATH=/path/to/python3.8
- no effect...
in the plugin repo, there is a copy of Pants script which determines the python version as following: https://github.com/pantsbuild/intellij-pants-plugin/blob/master/pants#L98-L130 so it takes into account 3.9 for 1.30 in my repo, I have the latest PANTS shell script, which is different!
Copy code
function set_supported_python_versions {
  local pants_version="$1"
  local pants_major_version
  local pants_minor_version
  pants_major_version="$(echo "${pants_version}" | cut -d '.' -f1)"
  pants_minor_version="$(echo "${pants_version}" | cut -d '.' -f2)"
  if [[ "${pants_major_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8')
    supported_python_versions_int=('36' '37' '38')
    supported_message='3.6, 3.7, or 3.8'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -eq 0 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8')
    supported_python_versions_int=('36' '37' '38')
    supported_message='3.6, 3.7, or 3.8'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.7' '3.8' '3.6')
    supported_python_versions_int=('37' '38' '36')
    supported_message='3.7, 3.8, or 3.6 (deprecated)'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -lt 5 ]]; then
    supported_python_versions_decimal=('3.8' '3.7')
    supported_python_versions_int=('38' '37')
    supported_message='3.7 or 3.8'
  else
    # We put 3.9 first because Apple Silicon only works properly with Python 3.9, even though it's possible to have
    # older Pythons installed. This makes it more likely that Pants will work out-of-the-box.
    supported_python_versions_decimal=('3.9' '3.8' '3.7')
    supported_python_versions_int=('39' '38' '37')
    supported_message='3.7, 3.8, or 3.9'
  fi
}
for some reason Python3.9 is missing there?
I double checked: it is indeed missing now for the latest official 1.30 distribution: https://pantsbuild.github.io/setup/pants
h
Yeah, 1.30 wasn't officially supported on Python 3.9
but it occurs to me that you are free to edit that script any way you like, to point it at any interpreter you like
you can completely override all that logic and just hardwire the interpreter you want
at least to test if that works...
s
I already tried to modify it. Out of the box, bundled venv version is not compatible with python 3.9
Copy code
Successfully installed pip-21.1.1
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 2.5.0.dev2, 2.5.0.dev3, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2)
ERROR: No matching distribution found for pantsbuild.pants==1.30.0
./pants: line 282: /home/serge/.cache/pants/setup/bootstrap-Linux-x86_64/1.30.0_py39/bin/python: No such file or directory
so probably it is not the best idea...
h
But maybe you could use changes to the pants script to force it to python3.8?
s
but the problem is that IDEA cannot into python3.8 🙂
and it ignores my pyenv
taking into account only system one (3.9 from Ubuntu 21.04...)
h
I thought the problem was that pants 1.30.x requires 3.6-3.8 but IDEA sees only 3.9
So what I hope will work is manipulating the PATH in the 
pants
script, since that is what IDEA runs when it executes pants
Does that make sense? Or am I misunderstanding the problem?
s
you understand it correctly. IDEA sees only 3.9, Pants 1.30.x requires 3.6-3.8
so I made the following changes to the script
Copy code
if [[ "${pants_major_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8', '3.9')
    supported_python_versions_int=('36' '37' '38', '39')
    supported_message='3.6, 3.7, 3.8 or 3.9'
now it does not work from the console
Copy code
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 2.5.0.dev2, 2.5.0.dev3, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3)
ERROR: No matching distribution found for pantsbuild.pants==1.30.0
./pants: line 282: /home/serge/.cache/pants/setup/bootstrap-Linux-x86_64/1.30.0_py39/bin/python: No such file or directory
IDEA fails as well (the same message in idea.log)
so, if I run IDEA from terminal session and not from IntelliJ Toolbox
Copy code
~/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-1/203.8084.24/bin
 % ./idea.sh
it seems to pick up the correct python version and project can be imported this issue was actually reported almost a year ago by a colleague of mine: https://github.com/pantsbuild/intellij-pants-plugin/issues/505
h
I think you need to eliminate 3.9 from that script entirely
Pants 1.30.x cannot resolve for 3.9
s
by default there is no 3.9
I added it manually. Then I reverted the script back to the original state (with only 3.8), ran IDEA from a terminal session and finally it was able to pick up proper pyenv version (3.8.6)
see the issue comment
9 Views
Open in Slack
PreviousNext
Powered by