https://pantsbuild.org/ logo
#general
Title
# general
s

square-oxygen-75288

05/04/2021, 11:10 AM
Hello! My system python version was updated to 3.9.x (because of Ubuntu 20.04 -> 21.04 upgrade) Now my IDEA complains about python version when I try to refresh the project:
Copy code
No valid Python interpreter found. For `pants_version = "1.30.0"`, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.
If I open IDEA's terminal and run
python
there, I will see:
Copy code
% python
Python 3.8.6 (default, May  4 2021, 09:31:45) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Cause I have pyenv configured for this (scala) project So I don't get how to get Pants in IDEA up and running cause it seems like it tries to use 3.9.x and fails (and it ignored pyenv) Any ideas how to fix it? Cheers.
h

happy-kitchen-89482

05/04/2021, 4:23 PM
Hi! Is this an issue with the Pants IDEA plugin?
It sounds like you need a 3.8 interpreter to be on IDEA's own PATH
how to set that is system-dependent, not sure how to do so for Ubuntu
Your experiment with IDEA's terminal might mean that that interpreter is already on the PATH, but maybe not?
Can you
echo $PATH
in that terminal?
s

square-oxygen-75288

05/06/2021, 8:37 AM
Hi! That's my $PATH in IDEA's terminal:
Copy code
% echo $PATH
/home/serge/bin:/home/serge/.nvm/versions/node/v14.16.1/bin:/home/serge/.pyenv/plugins/pyenv-virtualenv/shims:/home/serge/.pyenv/shims:/home/serge/.pyenv/bin:/home/serge/.cargo/bin:/home/serge/.local/bin:/home/serge/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
h

happy-kitchen-89482

05/06/2021, 2:37 PM
Thanks! So it looks like
python
in IntelliJ will most likely come from
/home/serge/.pyenv/shims
, which I suspect means it'll use the pyenv global interpreter (since IntelliJ is probably not running in a shell or a CWD in which you've set some other pyenv interpreter)
Which interpreter does
pyenv global
show?
If that is 3.9 then as a temporary workaround you could change the pyenv global interpreter to 3.8, until we figure out how the plugin selects an interpreter to use.
@wide-energy-11069 any ideas on this? (pun intended)
s

square-oxygen-75288

05/06/2021, 2:40 PM
Copy code
~ % pyenv global
3.8.6
the same in IDEA
h

happy-kitchen-89482

05/06/2021, 5:10 PM
Hrm
So it's now down to how the v1 intellij plugin locates its interpreter, which apparently is not via the regular IntelliJ PATH
@wide-energy-11069 is the expert here
w

wide-energy-11069

05/06/2021, 6:35 PM
Now my IDEA complains about python version when I try to refresh the project:
No valid Python interpreter found. For
pants_version = “1.30.0”
, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.
this should be related to how pants find the interpreter. is py36,37,38 on your system path?
Copy code
which -a python3
if not, make them available, and retry after restarting intellij. if still no dice and your pants is a pex, you can use /etc/pexrc to help it find it. E.g.
Copy code
$ cat /etc/pexrc
PEX_PYTHON_PATH=/opt/ee/python/2.7/bin/python2.7:/<some/path>/3.6/bin/python3.6
s

square-oxygen-75288

05/07/2021, 1:34 PM
checked out python3 locations:
Copy code
~ % which -a python3
/home/serge/.pyenv/shims/python3
/usr/bin/python3
/bin/python3
/home/serge/.pyenv/shims/python3
is 3.8.6 other 2 are 3.9.4
Copy code
% which -a python                 
/home/serge/.pyenv/shims/python
this one is also 3.8.6
w

wide-energy-11069

05/10/2021, 4:36 AM
iirc intellij won’t respect venv when running pants, so the right python needs to be on system path.
s

square-oxygen-75288

05/10/2021, 8:31 AM
I assume that changing
/bin/python3
and
/usr/bin/python3
from 3.9.x to 3.8.x can lead to some unforeseen problems with the operating system itself, cause all the packages will assume 3.9.x and not 3.8.x
w

wide-energy-11069

05/10/2021, 2:37 PM
so have you tried the /etc/pexrc approach?
s

square-oxygen-75288

05/10/2021, 5:55 PM
I really don't understand how to check if "my pants is a pex"
h

happy-kitchen-89482

05/10/2021, 5:58 PM
What happens if you put
PEX_PYTHON_PATH=/path/to/python3.8
in
/etc/pexrc
?
Oh, and it may be that 1.30.x doesn't handle pyenv shims well. You could try adding the path to the real underlying interpreter (e.g.,
/Users/<username>/.pyenv/versions/3.8.6/bin/python
) to intellij's PATH and restart intellij.
I vaguely remember 1.30 having issues with pyenv shims
s

square-oxygen-75288

05/10/2021, 6:09 PM
but I don't have
/etc/pexrc
h

happy-kitchen-89482

05/10/2021, 8:08 PM
Can you create it?
It's probably not that, but can't hurt to rule it out.
s

square-oxygen-75288

05/11/2021, 11:51 AM
I created it
Copy code
~ % cat /etc/pexrc 
PEX_PYTHON_PATH=/home/serge/.pyenv/versions/3.8.6/bin/python3.8
but it did not help
h

happy-kitchen-89482

05/11/2021, 9:42 PM
Right, so I think the important thing is to put the "real" path to the 3.8 interpreter (not the shim) on IntelliJ's PATH
I am not sure how to set an env var for a GUI app in MacOS though
but I assume there is a way
@wide-energy-11069 What sets the Intellij binary's PATH?
w

wide-energy-11069

05/11/2021, 9:56 PM
it just takes the system PATH + ~/.bashXXX loaded for terminal
PEX_PYTHON_PATH=/home/serge/.pyenv/versions/3.8.6/bin/python3.8
in case this doesn’t work. this might have to be path to real py38 interpreter, not in venv
also running pex with a venv probably isn’t a good idea - if there are already libraries installed in venv, it may conflict with whatever the pex brings in.
h

happy-kitchen-89482

05/11/2021, 10:33 PM
I don't think pex is relevant here?
This is the actual PATH we need to affect?
w

wide-energy-11069

05/12/2021, 6:04 AM
/etc/pexrc
is relevant for pex (assuming the pants run here is in a pex), especially if the right interpreter for pex isn’t available on system PATH.
h

happy-kitchen-89482

05/12/2021, 6:20 AM
I don't think the Pants run here is a pex? I assume it's installed via a runner script from the wheels
w

wide-energy-11069

05/12/2021, 6:37 PM
oh ok. in case that please ignore pexrc
h

happy-kitchen-89482

05/12/2021, 7:01 PM
How does one set env vars on GUI-launched applications like IntelliJ on OSX?
w

wide-energy-11069

05/13/2021, 2:50 AM
ah yeah, it’s available in the run configurations.
so the downside is that this needs to be configured manually every time, or requires a custom plugin to overwrite every time
s

square-oxygen-75288

05/13/2021, 9:25 AM
but how do I set an env variable for Pants IDEA plugin? I cannot even import a project.
w

wide-energy-11069

05/13/2021, 7:58 PM
@happy-kitchen-89482 I guess there could be a larger question in general. If Pants were to run where the right interpreter isn’t on the system PATH, should there be a way to tell Pants how to find it, similar to /etc/pexrc for pex. Could be a separate env var that only PANTS will use e.g.
PANTS_INTERPRETOR_PATH
. just throwing this idea out.
h

happy-kitchen-89482

05/13/2021, 8:19 PM
There are ways for pants in general, what I'm not clear on is how IntelliJ invokes pants
does it call the
./pants
wrapper script in the repo?
because it just occurs to me that you can do whatever PATH manipulations you want in that script...
w

wide-energy-11069

05/14/2021, 4:39 AM
yes, it calls
./pants
exactly, and that’s how twitter does it. so
./pants
is essentially a wrapper which eventually calls the pants.pex in our case
s

square-oxygen-75288

05/14/2021, 8:29 AM
I tried tinkering with IDEA Absolute Path Variables restarting it every time added the whole PATH there - no effect
Copy code
/home/serge/bin:/home/serge/.nvm/versions/node/v14.16.1/bin:/home/serge/.pyenv/plugins/pyenv-virtualenv/shims:/home/serge/.pyenv/shims:/home/serge/.pyenv/bin:/home/serge/.cargo/bin:/home/serge/.local/bin:/home/serge/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
added only
PATH=/home/serge/.pyenv/versions/3.8.6/bin
- no effect added only
PEX_PYTHON_PATH=/path/to/python3.8
- no effect...
in the plugin repo, there is a copy of Pants script which determines the python version as following: https://github.com/pantsbuild/intellij-pants-plugin/blob/master/pants#L98-L130 so it takes into account 3.9 for 1.30 in my repo, I have the latest PANTS shell script, which is different!
Copy code
function set_supported_python_versions {
  local pants_version="$1"
  local pants_major_version
  local pants_minor_version
  pants_major_version="$(echo "${pants_version}" | cut -d '.' -f1)"
  pants_minor_version="$(echo "${pants_version}" | cut -d '.' -f2)"
  if [[ "${pants_major_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8')
    supported_python_versions_int=('36' '37' '38')
    supported_message='3.6, 3.7, or 3.8'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -eq 0 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8')
    supported_python_versions_int=('36' '37' '38')
    supported_message='3.6, 3.7, or 3.8'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.7' '3.8' '3.6')
    supported_python_versions_int=('37' '38' '36')
    supported_message='3.7, 3.8, or 3.6 (deprecated)'
  elif [[ "${pants_major_version}" -eq 2 && "${pants_minor_version}" -lt 5 ]]; then
    supported_python_versions_decimal=('3.8' '3.7')
    supported_python_versions_int=('38' '37')
    supported_message='3.7 or 3.8'
  else
    # We put 3.9 first because Apple Silicon only works properly with Python 3.9, even though it's possible to have
    # older Pythons installed. This makes it more likely that Pants will work out-of-the-box.
    supported_python_versions_decimal=('3.9' '3.8' '3.7')
    supported_python_versions_int=('39' '38' '37')
    supported_message='3.7, 3.8, or 3.9'
  fi
}
for some reason Python3.9 is missing there?
I double checked: it is indeed missing now for the latest official 1.30 distribution: https://pantsbuild.github.io/setup/pants
h

happy-kitchen-89482

05/14/2021, 7:08 PM
Yeah, 1.30 wasn't officially supported on Python 3.9
but it occurs to me that you are free to edit that script any way you like, to point it at any interpreter you like
you can completely override all that logic and just hardwire the interpreter you want
at least to test if that works...
s

square-oxygen-75288

05/14/2021, 7:12 PM
I already tried to modify it. Out of the box, bundled venv version is not compatible with python 3.9
Copy code
Successfully installed pip-21.1.1
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 2.5.0.dev2, 2.5.0.dev3, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2)
ERROR: No matching distribution found for pantsbuild.pants==1.30.0
./pants: line 282: /home/serge/.cache/pants/setup/bootstrap-Linux-x86_64/1.30.0_py39/bin/python: No such file or directory
so probably it is not the best idea...
h

happy-kitchen-89482

05/14/2021, 7:26 PM
But maybe you could use changes to the pants script to force it to python3.8?
s

square-oxygen-75288

05/14/2021, 7:28 PM
but the problem is that IDEA cannot into python3.8 🙂
and it ignores my pyenv
taking into account only system one (3.9 from Ubuntu 21.04...)
h

happy-kitchen-89482

05/14/2021, 9:57 PM
I thought the problem was that pants 1.30.x requires 3.6-3.8 but IDEA sees only 3.9
So what I hope will work is manipulating the PATH in the
pants
script, since that is what IDEA runs when it executes pants
Does that make sense? Or am I misunderstanding the problem?
s

square-oxygen-75288

05/15/2021, 4:56 PM
you understand it correctly. IDEA sees only 3.9, Pants 1.30.x requires 3.6-3.8
so I made the following changes to the script
Copy code
if [[ "${pants_major_version}" -eq 1 ]]; then
    supported_python_versions_decimal=('3.6' '3.7' '3.8', '3.9')
    supported_python_versions_int=('36' '37' '38', '39')
    supported_message='3.6, 3.7, 3.8 or 3.9'
now it does not work from the console
Copy code
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 2.5.0.dev2, 2.5.0.dev3, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3)
ERROR: No matching distribution found for pantsbuild.pants==1.30.0
./pants: line 282: /home/serge/.cache/pants/setup/bootstrap-Linux-x86_64/1.30.0_py39/bin/python: No such file or directory
IDEA fails as well (the same message in idea.log)
so, if I run IDEA from terminal session and not from IntelliJ Toolbox
Copy code
~/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-1/203.8084.24/bin
 % ./idea.sh
it seems to pick up the correct python version and project can be imported this issue was actually reported almost a year ago by a colleague of mine: https://github.com/pantsbuild/intellij-pants-plugin/issues/505
h

happy-kitchen-89482

05/15/2021, 6:49 PM
I think you need to eliminate 3.9 from that script entirely
Pants 1.30.x cannot resolve for 3.9
s

square-oxygen-75288

05/15/2021, 7:26 PM
by default there is no 3.9
I added it manually. Then I reverted the script back to the original state (with only 3.8), ran IDEA from a terminal session and finally it was able to pick up proper pyenv version (3.8.6)
see the issue comment
9 Views