https://pantsbuild.org/ logo
#general
Title
# general
b

billions-spring-9652

12/19/2021, 4:44 PM
Howdy. I'm trying to run pants 2.8.0 with
interpreter_constraints = ["CPython==3.10.*"]
. I'm on ubuntu 18.04 (bionic) and my python3.10 binary is from the deadsnakes ppa. This is what I get:
ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.10.*' failed with exit code 102.
stdout:
stderr:
Could not find a compatible interpreter.
Examined the following interpreters:
1.) /usr/bin/python2.7 CPython==2.7.17
2.) /usr/bin/python3.6 CPython==3.6.9
3.) /usr/bin/python3.8 CPython==3.8.0
4.) /usr/bin/python3.9 CPython==3.9.9
(That 3.9.9 version is from deadsnakes as well. If I specify CPython==3.9.*, it's picked up and everything's fine). So, is my python3.10 installation broken somehow, or not supported?
f

fast-nail-55400

12/19/2021, 4:52 PM
What is the path to that 3.10 interpreter? And what is the search path set to in
pants.toml
? From that output, it seems like Pants did not even find the
python
binary.
b

billions-spring-9652

12/19/2021, 4:56 PM
/usr/bin/python3.10, and the current pants.toml is a stripped down one that doesn't specify interpreter_search_paths. I had one that specified
["<PYENV>", "/usr/local/bin", "/usr/bin"]
earlier. Same effect.
e

enough-analyst-54434

12/19/2021, 5:02 PM
Pants 2.8.0 uses Pex 2.1.54 (2.8.0rc2 had the upgrade cherry pick). Pex 2.1.55 is the first release with official Python 3.10 support: https://github.com/pantsbuild/pex/blob/main/CHANGES.rst#2155 I'm surprised Pex 2.1.54 is failing to identify a python 3.10 interpreter, but even if that worked, later use to build wheels from sdists would have failed anyhow (This vendored setuptools patch was needed: https://github.com/pantsbuild/setuptools/pull/1).
So, short answer, is this is a no go on 2.8.0. I can work on a Pex upgrade cherry-pick though for 2.8.1rc2 though later today if this is critical to have before 2.9.0 (we just released 2.9.0rc0).
b

billions-spring-9652

12/19/2021, 5:10 PM
An rc is ok for me. But I just changed to
pants_version = "2.9.0rc0"
, and I get the same thing. I suppose there's something specific going on with my setup.
These deadsnakes python versions are newly installed, and haven't seen any real use yet. This is a build server with lots of stuff installed, so there are opportunities for things to go wrong.
Oh. I'm playing around in the process-execution directory, and I've discovered something interesting.
python3 ./pex --interpreter-constraint $'CPython==3.10.*' -- -c 'print(42)'
gives the familiar constraint failure, but if I specifically invoke python3.10, that is:
python3.10 ./pex --interpreter-constraint $'CPython==3.10.*' -- -c 'print(42)'
, that works!
e

enough-analyst-54434

12/19/2021, 6:20 PM
Do the dead snakes pythons rely on a custom LD_LIBRARY_PATH per chance?
Likely more on point, what exact packages do you install from deadsnakes? Reading here: https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa it looks like Debian-style "many packages composing a full install" is maintained. You may need the -distutils and -venv packages too.
I'll see if I can repro this in a Bionic container here in a bit.
b

billions-spring-9652

12/19/2021, 8:07 PM
I don't believe there's any LD_LIBRARY_PATH trickery going on with the deadsnakes packages. The packages I installed initially are python3.10 and python3.10-venv (which also pulls in python3.10-distutils). I didn't have python3-distlib initially, but now I do.
I've started on a Dockerfile to reproduce this myself, so you may want to hold off on that.
All right. So the official ubuntu:18.04 image has the same problem.
e

enough-analyst-54434

12/19/2021, 9:06 PM
I repro:
Copy code
$ cat Dockerfile 
FROM ubuntu:18.04

RUN apt-get update && \
    apt-get install -y software-properties-common && \
    add-apt-repository -y ppa:deadsnakes/ppa

RUN apt-get update && \
    apt-get install -y \
    curl \
    python \
    python-dev \
    python3.8 \
    python3.8-dev \
    python3.8-venv \
    python3.8-distutils \
    python3.9 \
    python3.9-dev \
    python3.9-venv \
    python3.9-distutils \
    python3.10 \
    python3.10-dev \
    python3.10-venv \
    python3.10-distutils

$ docker build -t bionic:deadsnakes-310 .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM ubuntu:18.04
 ---> 5a214d77f5d7
Step 2/3 : RUN apt-get update &&     apt-get install -y software-properties-common &&     add-apt-repository -y ppa:deadsnakes/ppa
 ---> Using cache
 ---> b1b5a578052e
Step 3/3 : RUN apt-get update &&     apt-get install -y     curl     python     python-dev     python3.8     python3.8-dev     python3.8-venv     python3.8-distutils     python3.9     python3.9-dev     python3.9-venv     python3.9-distutils     python3.10     python3.10-dev     python3.10-venv     python3.10-distutils
 ---> Using cache
 ---> 29464760100f
Successfully built 29464760100f
Successfully tagged bionic:deadsnakes-310
$ docker run --rm -it bionic:deadsnakes-310 bash -i
root@f4f52262c693:/# curl -sSL <https://github.com/pantsbuild/pex/releases/download/v2.1.56/pex> > pex-2.1.56 
root@f4f52262c693:/# chmod +x pex-2.1.56 
root@f4f52262c693:/# ./pex-2.1.56 --interpreter-constraint "==3.11.*"         
Could not find a compatible interpreter.

Examined the following interpreters:
1.) /usr/bin/python2.7 CPython==2.7.17
2.) /usr/bin/python3.6 CPython==3.6.9
3.) /usr/bin/python3.8 CPython==3.8.12
4.) /usr/bin/python3.9 CPython==3.9.9

No interpreter compatible with the requested constraints was found:
  Version matches ==3.11.*
root@f4f52262c693:/#
I'll dig here a bit. Thanks for the data @billions-spring-9652.
Yeah - somehow Pex is not seeing that interpreter on the $PATH even though it is:
Copy code
root@f4f52262c693:/# rm -rf ~/.pex/
root@f4f52262c693:/# ./pex-2.1.56 -vvvvvvvvvv --interpreter-constraint "==3.11.*"
pex: Spawning a maximum of 16 parallel jobs to process:
  /usr/bin/python
  /usr/bin/python2
  /usr/bin/python2.7
  /usr/bin/python3
  /usr/bin/python3.6
  /usr/bin/python3.6m
  /usr/bin/python3.8
  /usr/bin/python3.9
pex: Hashing pex: 20.0ms
...
b

billions-spring-9652

12/19/2021, 9:11 PM
You've got 3.11 in there though.
e

enough-analyst-54434

12/19/2021, 9:11 PM
Yeah - I was smoking out the full list by asking Pex for something I know doesn;t exist.
b

billions-spring-9652

12/19/2021, 9:12 PM
ah, I see
e

enough-analyst-54434

12/19/2021, 9:12 PM
Pex finds 3.10 in GH actions CI ... so this will be interesting.
b

billions-spring-9652

12/19/2021, 9:15 PM
It'll be one of those debian python packaging weirdnesses, I guess.
e

enough-analyst-54434

12/19/2021, 9:15 PM
We'll see. Follow along here: https://github.com/pantsbuild/pex/issues/1544
b

billions-spring-9652

12/19/2021, 9:18 PM
All right. Thanks for helping me out.
e

enough-analyst-54434

12/19/2021, 9:23 PM
It ... its embarrassing. A fix for this will make it into 2.9.0rc1.
b

billions-spring-9652

12/19/2021, 9:32 PM
Hehe, ok 🙂
8 Views