https://pantsbuild.org/ logo
Join Slack
Powered by
Hm.. is there any expected cause for getting the f...
# general
r
Hm.. is there any expected cause for getting the following error while my 
asdf
setup has 
3.10.9
, 
3.11.4
, 
3.11.6
, and 
3.12.0
?:
Copy code
> pants lint ::
20:07:29.73 [ERROR] 1 Exception encountered:

Engine traceback:
  in `lint` goal

ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.11.6' failed with exit code 102.
stdout:

stderr:
Could not find a compatible interpreter.

Interpreters were found but they all appear to be broken:
1.) /home/joongi/.asdf/installs/python/3.12.0/bin/python3.12:
Traceback (most recent call last):
  File "<string>", line 4, in <module>
  File "/home/joongi/.cache/pants/named_caches/pex_root/isolated/3754108aac57ec0c996f3414782585924aee601c/pex/atomic_directory.py", line 23, in <module>
    from pex.third_party import attr
ImportError: cannot import name 'attr' from 'pex.third_party' (/home/joongi/.cache/pants/named_caches/pex_root/isolated/3754108aac57ec0c996f3414782585924aee601c/pex/third_party/__init__.py)

(See <https://github.com/pantsbuild/pex/issues/1027> for a list of known breaks and workarounds.)
it used to work until yesterday (?!) and suddenly began to crash... T_T
my pants.toml has the followings (
pyenv
is empty now as i'm testing migration to 
asdf
):
Copy code
[python]
enable_resolves = true
interpreter_constraints = ["CPython==3.11.6"]
tailor_pex_binary_targets = false

[python-bootstrap]
search_path = ["<PYENV>", "<ASDF>"]
Copy code
> asdf list python
  3.10.9
  3.11.4
  3.11.6
 *3.12.0
the pants version is 2.17.0
Q1. Why is it trying Python 3.12.0 although the interpreter constraints have only 3.11.6? Q2. Even if I run 
asdf local python 3.11.6
, it still gets the same error with Python 3.12.0. It only works when I run 
asdf global python 3.11.6
...
If I completely remove all 
$HOME/.tool-versions
and 
BUILD_ROOT/.tool-versions
, it works. But I get "No version is set for command python" when I try to run 
python
....
pants seems to search in the following precedence: 1. try "asdf-local" version (
BUILD_ROOT/.tool-versions
) 2. try "asdf-global" version (
$HOME/.tool-versions
) 3. try all available versions in 
asdf list python
but there are some edge cases that fails to continue on the subsequent search targets or correctly match the interpreter compatibility
it seems to get stuck in the step 2 (asdf-global)
e
Modern Pex supports 3.12 - are you using modern Pex?
Looks like 2.1.139 or newer is needed for 3.12 support: https://pypi.org/project/pex/2.1.139/#data
r
i'm just relying on the pex that comes with pants 2.17.0, without specifying any special options for pex
before thinking about pex's python 3.12 support, i think pants should choose 3.11.6 and there should be no problem in the first place.
maybe, does the interpreter compatibility check procedure rely on pex's ability to run on a specific version of python interpreter?
currently i'm working around this by explicitly prepending 
<ASDF_LOCAL>
before 
<ASDF>
in the interpreter search target list and setting 
asdf local python 3.11.6
(https://github.com/lablup/backend.ai/pull/1608)
afaik, 
<ASDF_LOCAL>
is "included" in 
<ASDF>
, so this shouldn't be necessary..
e
I can't speak for Pants / your battles there, but if you want instructions for upgrading Pex if you can't figure out the needed 
[pex-cli]
options, speak up.
r
https://github.com/lablup/backend.ai/blob/main/pants.toml#L79-L88 well, actually i used to write it before..... 🤣 ...
anyway, thanks for teaching me that i eventually need to manually upgrade the pex version to migrate to Python 3.12
e
You have the wrong idea about Pants I think. For the most part it lets Pex do the picking. If the interpreter used to run the Pex CLI does not match constraints, it searches further (not Pants). It hits 3.12 and dies.
Pex is highly backward compatible and takes pride in keeping that guarantee. So you really can just upgrade Pex in general (unlike Pants).
r
hm.... in my perspective, i thought that the interpreter search process was independent from the operation of pex
i'll try upgrading pex
ok, now pex itself does not "crash", but still pants does not continue to search on the next search targets (2 → 3)
e
Pants implements 99% of the Python backend on Pex.
Ok, at least now you have a clean Pants bug to report to those folks.
✅ 1
Exit code 102 does look like a pretty clear signal / lead from your 3.11.6 interpreter though.
r
my expectation on the asdf support was that it should work in the same way the pyenv support works, but hm....
e
It was community contributed IIRC and underwent less scrutiny than the earlier pyenv support work.
Maybe you can make it better.
r
yeah, i tried to read the pyenv/asdf interpreter search codes, and they seem to be quite different
i currently don't have enough time to debug by myself, but someday i could try.
l
Hitting this myself, taking a look under the hood
Open in Slack
PreviousNext
Powered by