What is the correct way to use a different version of Python in CI on GitHub self hosted runners? I'm using GitHub actions with pants 2.22.0 and I want to use Python 3.12 which we use locally and in our current polyrepo setup. Trying to use 3.12 fails with

home/runner/_work/_tool/Python/3.12.7/x64/bin/python3.12: error while loading shared libraries: libpython3.12.so.1.0: cannot open shared object file: No such file or directory

I think this is related to https://github.com/actions/setup-python/issues/871 Are there any accepted solutions beyond the workaround in that ticket to do this that people are using? We have the following in

pants.toml

[python]
interpreter_constraints = ["==3.12.*"]

pants lint ::

in CI is failing with

Error: 2.79 [ERROR] 1 Exception encountered:
Engine traceback:
  in `lint` goal
ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.12.*' failed with exit code 102.
stdout:
stderr:
Could not find a compatible interpreter.
Examined the following working interpreters:
1.) /usr/bin/python3.10 CPython==3.10.12
Skipped the following broken interpreters:
1.) /home/runner/_work/_tool/Python/3.12.7/x64/bin/python3.12:
/home/runner/_work/_tool/Python/3.12.7/x64/bin/python3.12: error while loading shared libraries: libpython3.12.so.1.0: cannot open shared object file: No such file or directory

There seems to be a fix at https://github.com/actions/python-versions/pull/275.

that fix has been sitting for 5 months - I'm not optimistic it's ever getting merged.

Would something like the Pants https://github.com/pantsbuild/actions/tree/main/expose-pythons help?

Also, you are using a self-hosted runner, could you not just setup the various Python versions in your own runner's image?

I might have to resort to that.

for example, the Pants CI for Linux/ARM build uses a custom AMI built as per https://github.com/pantsbuild/pants/blob/main/docs/docs/contributions/releases/github-actions-linux-aarch64-runners.mdx

Hi Wayne, I had a similar problem with GitHub hosted runners and the

setup-python

action. Adding the following to my

pants.toml

allowed the required environment variable to propagate through to the pants processes:

[subprocess-environment]
env_vars = ["LD_LIBRARY_PATH"]

based on the comment - could you try, at the job level, to set

AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache

and/or

RUNNER_TOOL_CACHE=/opt/hostedtoolcache

so that the self-hosted runner matches the hosted runner config?

with

env:
  AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache

Setup Python fails with

Error: mkdir: cannot create directory '/opt/hostedtoolcache': Permission denied

I can add a previous step to create this and chown it to root, and setup python then succeeds. But I don't understand the potential security concerns having done this.

- name: Create directory and set permissions
        run: |
          sudo mkdir -p /opt/hostedtoolcache
          sudo chmod -R 777 /opt/hostedtoolcache