https://pantsbuild.org/ logo
Join Slack
Powered by
I'm having trouble setting the python executable s...
# general
v
I'm having trouble setting the python executable search path to a flox installed python (wrapper around Nix)
Copy code
[python-bootstrap]
search_path = [
    "%(env.FLOX_ENV)s/bin"
]
pants test, run
fails but 
pants export --py-resolve-format=symlinked_immutable_virtualenv
works fine. I also tried hardcoding the value of env.FLOX_ENV and it still didn't work. But using
Copy code
[python-bootstrap]
search_path = [
    "<PATH>"
]
works just fine. The error I get is "No interpreter compatible with the requested constraints was found:"
f
And what interpreter constrains have you set in 
pants.toml
or 
BUILD
files? And what version of Python is stored at 
$FLOX_ENV/bin
. Pants and/or Pex probably sees the flox Python but is rejecting it as incompatible.
v
they're both the same:
Copy code
interpreter_constraints = ["==3.13.*"]
Copy code
/Users/mfairley/jazmoai/jazmo/.flox/run/aarch64-darwin.jazmo.dev/bin/python --version
Python 3.13.3
I also think it's the only version available on my PATH so if it's not taking it from flox when using <PATH> then I don't know where it's coming from.
f
That errors comes from pex so increase Pex verbosity.
You can also preserve the execution sandbox with 
--keep-sandboxes=on_failure
and manually run the pex command which is not accepting the Python.
Pex should print out the reason why it rejected the Python.
v
Thanks! I do see
Copy code
20:12:15.98 [INFO] pex: Resolving interpreters
pex: Resolving interpreters :: Searching for pyenv root...
pex: No pyenv installation was found.
I do have pyenv installed although I'm not using it in my flox environment. This suggests that pex is still relying on pyenv though. Perhaps I'm missing a setting somewhere to tell it to look for python elsewhere
I don't see such an option. How do I get pex to directly use python in the path instead of using pyenv?
f
Pants shouldn't be configuring pyenv unless 
<PYENV>
was included in search path. Let's verify your 
[python-bootstrap].search_path
option's actual value. Run 
pants help-advanced python-bootstrap
What does it say the value of 
search_path
is? And maybe that error is skipped? If you set 
PEX_VERBOSE=9
you should get more logging (if invoking a preserved sandbox)
v
I'm seeing this:
Copy code
search_path
      default: [
          "<PYENV>",
          "<ASDF>",
          "<PATH>"
      ]
      current value: [
          "/Users/mfairley/jazmoai/jazmo/.flox/run/aarch64-darwin.jazmo.dev/bin"
      ] (pants.toml)
f
Are you running Pants with 
-ldebug
as well?
Frankly you should be getting a message detailing each Python on the search path and why it was rejected. A message starting with 
Examined the following interpreters
v
I'm seeing 
No interpreters could be found on the system.
f
What's output of 
ls -l /Users/mfairley/jazmoai/jazmo/.flox/run/aarch64-darwin.jazmo.dev/bin
v
lrwxr-xr-x 1 root wheel 62 Dec 31 1969 /Users/mfairley/jazmoai/jazmo/.flox/run/aarch64-darwin.jazmo.dev/bin -> /nix/store/1a8xg8l3m67hxinxzzcsak9736qm9vsf-python3-3.13.3/bin
f
And if you go through the symlink, what is the contents of the 
bin
directory?
v
Copy code
flox [jazmo] ➜  jazmo git:(main) ✗ ls /nix/store/1a8xg8l3m67hxinxzzcsak9736qm9vsf-python3-3.13.3/bin
idle              idle3             idle3.13          pydoc             pydoc3            pydoc3.13         python            python-config     python3           python3-config    python3.13        python3.13-config
f
Try adding augmenting search path with the name of the python binary (e.g., 
python3.13
)
so path to the file instead of the directory
try to rule out symlink issues
and can you share full output from running the failing pex command?
and full pex command (from the 
__run.sh
file in the sandbox)
v
Ok, using the full path didn't work either:
Copy code
search_path = [
    "/nix/store/1a8xg8l3m67hxinxzzcsak9736qm9vsf-python3-3.13.3/bin/python3.13"
]
Copy code
#!/usr/bin/env bash
# This command line should execute the same process as pants did internally.
cd /private/var/folders/lf/tr4gz0k514lf165m8gcg73yw0000gn/T/pants-sandbox-Bf8jUX
env -i CPPFLAGS='' LDFLAGS='' PATH=$'/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin' PEX_IGNORE_RCFILES=true PEX_PYTHON=/pants-named-caches/python_build_standalone/aa16a7a87faf96d6f8c534a20d60da226da7ea3319cb2a376363be3dbd48baba/bin/python3 PEX_ROOT=.cache/pex_root /pants-named-caches/python_build_standalone/aa16a7a87faf96d6f8c534a20d60da226da7ea3319cb2a376363be3dbd48baba/bin/python3 ./pex --tmpdir .tmp -vvvvvvvvv --no-emit-warnings --pip-version 24.2 --python-path /nix/store/1a8xg8l3m67hxinxzzcsak9736qm9vsf-python3-3.13.3/bin/python3.13 --interpreter-constraint $'CPython==3.13.*' -- -c $'import hashlib, os, sys\n\npython = os.path.realpath(sys.executable)\nprint(python)\n\nhasher = hashlib.sha256()\nwith open(python, "rb") as fp:\n  for chunk in iter(lambda: fp.read(8192), b""):\n      hasher.update(chunk)\nprint(hasher.hexdigest())\n'
this command works fine 
env -i PATH=/nix/store/1a8xg8l3m67hxinxzzcsak9736qm9vsf-python3-3.13.3/bin python
f
And if you add 
PEX_VERBOSE=9
to that command in 
__run.sh
and run it, what's the full output?
and what Pants and Pex versions are you using?
probably good to try with an up-to-date version of Pex
v
Copy code
bash __run.sh 
env: /pants-named-caches/python_build_standalone/aa16a7a87faf96d6f8c534a20d60da226da7ea3319cb2a376363be3dbd48baba/bin/python3: No such file or directory
Copy code
pants_version = "2.26.1"
how would I find the pex version that pants is using?
f
pants help pex-cli
will print the current configured Pex version.
v
hmm doesn't seem to print a version
f
are you using 
docker_environment
?
v
yes
f
You need the applicable Python installed inside the container.
v
oh, does it need to be installed at the same path within the container?
f
no because you can override options inside the environent
set 
python_boostrap_search_path
as needed on the 
docker_environment
target
(most options have a corresponding field on environment targets to allow environment-specific overrides)
also it would have been helpful to share that context of using 
docker_environment
.... I only noticed because I wrote the 
docker_environment
support and know that the 
/pants-named-caches
directory is where the Docker volume is mounted in the container ...
v
Ok, sorry about that. I'm new to pants so don't quite understand all the mechanisms yet. Didn't realize this search path is also applied inside the docker_environment. Switching to local environment did indeed fix the issue
f
v
awesome, thank you!
f
> Ok, sorry about that. I'm new to pants so don't quite understand all the mechanisms yet. Didn't realize this search path is also applied inside the docker_environment. Switching to local environment did indeed fix the issue No worries. Just a plea for more context to be provided. 🙂
👍 1
14 Views
Open in Slack
PreviousNext
Powered by