#23269 Weird GitHub runner `ProcessExecutionFailure` failures due to cache conflict Issue created by originalrkk Describe the bug Trying to diagnose a strange issue where

pants lint

CI on our self-hosted GitHub runner occasionally fails with this error:

16:01:50.21 [INFO] Completed: Building pylint_runner.pex
16:01:50.21 [INFO] Completed: Scheduling: Building pylint_runner.pex
Error: 0.22 [ERROR] 1 Exception encountered:

Engine traceback:
  in `lint` goal

ProcessExecutionFailure: Process 'Building pylint_runner.pex' failed with exit code 1.
stdout:

stderr:
[Errno 17] File exists: '../../../../../../../../installed_wheels/2/b595<REDACTED LONG HASH 1>/pyfarmhash-0.4.0-cp311-cp311-linux_x86_64.whl/pyfarmhash-0.4.0.dist-info/LICENSE' -> '/users/<REDACTED RUNNER USER>/.cache/pants/named_caches/pex_root/venvs/3/4f22<REDACTED SHORT HASH 1>/d942<REDACTED SHORT HASH 2>.lck.work/lib/python3.11/site-packages/pyfarmhash-0.4.0.dist-info/LICENSE'


error: Recipe `lint` failed on line 315 with exit code 1

Some things to note: • This happens even after we punt runners, clear caches. • It's always

pyfarmhash

and always about the

LICENSE

file. • It doesn't always occur, and we haven't been able to replicate locally. About our dependencies and versioning: We have a pyproject.toml file. We run

poetry lock

, and Poetry processes this into a poetry.lock file. We then

poetry export -f constraints.txt -o poetry_constraints.txt

and

pants --no-pantsd --no-local-cache generate-lockfiles

. (This allows us to use Poetry shells as well as Pants, which makes development life a little easier.) The relevant sections in pants.toml look like:

[python]
enable_resolves = true
resolves = { python-default = "python-default.lock" }
resolves_to_constraints_file = { python-default = "poetry_constraints.txt" }

[pylint]
install_from_resolve = "python-default"

Now, we recently updated several dependencies in our monorepo. Pants went from 2.27.0 to 2.31.0, pylint went from 3.3.7 to 4.0.5, and types-pyfarmhash went from 0.4.0.20240902 to 0.4.0.20260408. pyfarmhash itself was not updated, however. Worth noting that pyfarmhash is part of the handful of

module_mapping

name we have in our root BUILD file:

poetry_requirements(
    name = "poetry",
    module_mapping = {
        ...
        "pyfarmhash": ("farmhash",),
    },
)

It also appears in .pylintrc:

[MAIN]
...
extension-pkg-allow-list=farmhash

As for the CI portion, Pants is installed during the GitHub workflow like so (we probably should move Pants installation into our runner Dockerfile, but haven't gotten around to it yet):

- name: Install Pants
      uses: pantsbuild/actions/init-pants@v10
      with:
        gha-cache-key: gh
        named-caches-hash: ${{ hashFiles('python-default.lock') }}
        setup-python-for-plugins: 'true'

And here is our whole pants.ci.toml (trying to think of anything that might be marginally relevant):

[GLOBAL]
remote_cache_read = true
remote_cache_warnings = "always"
remote_cache_write = true
remote_store_address = "grpc://<REDACTED>.bazel.<REDACTED>:<REDACTED PORT>"
remote_instance_name = "<REDACTED>"

Pants version This is Pants 2.31.0 (see above). OS This is exclusively Linux, and it only happens on our GitHub self-hosted runners (built with

FROM summerwind/actions-runner:ubuntu-22.04

). Additional info Happy to provide anything else you need. pantsbuild/pants