#17405 Running Python code in `experimental_shell_command` requires building a PEX and specifying an appropriate interpreter as a tool New issue created by huonw Describe the bug When

experimental_shell_command

needs to run Python code from the repo, it seems the best way to do that is to depend on a PEX and then specify a compatible Python version as a

tool

(and

bash

too). This seems... unwieldy, and the second part in particular seems like it's liable to end up with 'works on my machine' problems, if developers have different

PATH

configurations. Example: https://gist.github.com/huonw/47bc63951eac7a05a3a3442843f040a9

git clone <https://gist.github.com/47bc63951eac7a05a3a3442843f040a9.git>
cd 47bc63951eac7a05a3a3442843f040a9
./pants run //:print

This would be better if (numbers corresponding to

BUG

comments): 1. (convenience) the shell command could just depend on

script.py

(and its venv/other dependencies) directly, similar to how

./pants run script.py

works mostly the same as

./pants run //:pex

. This would avoid needing another PEX binary that gets unnecessarily explicitly packaged on

./pants package ::

. 2. (reliability/system dependencies) we didn't have to specify both a compatible Python and Bash in

tools

3. (reliability/system dependencies) related to that, if I use

interpreter_constraints = ["CPython==3.7.*"]

(and

tools=["python3.7", "bash"]

), running

./pants run script.py

works fine, but neither

./pants run //:pex

nor

./pants run //:print

do. • I have 3.7.13 installed via pyenv, but not on my path by default (i.e. running

python3.7

hits the pyenv shim, and gets an error like ``The `python3.7' command exists in these Python versions: 3.7.13``). • Running

pyenv global 3.7.13

or

pyenv shell 3.7.13

first, and then rerunning

./pants run //:print

works fine, but would presumably break anything using other versions... Key files for posterity/convenience:

# script.py
print("hello")

# BUILD
python_sources(name="py")
pex_binary(name="pex", entry_point="script.py")

experimental_shell_command(
    name="shell",
    command="{chroot}/pex.pex > output.txt",
    tools=["python3.9", "bash"], # BUG 2
    outputs=["output.txt"],
    dependencies=[":pex"], # BUG 1
)

experimental_run_shell_command(
    name="print", command="cat {chroot}/output.txt", dependencies=[":shell"]
)

# pants.toml
[GLOBAL]
pants_version = "2.15.0a0"

backend_packages = [
  "pants.backend.shell",
  "pants.backend.python",
]

[python]
interpreter_constraints = ["CPython==3.9.*"] # BUG 3

[anonymous-telemetry]
enabled = false

Pants version 2.15.0a0 OS macOS Additional info This is related to https://pantsbuild.slack.com/archives/C046T6T9U/p1666315386420799, and has the same context as #17345: we're using

experimental_shell_command

for ad hoc code generation, where we execute part of our API code to generate a schema file. pantsbuild/pants