We're setting up Depot Cache (REAPI) for remote caching in CI. We've gotten lint/check/test cache keys stable by removing rotating credentials from

[subprocess-environment].env_vars

. However, we're stuck on

pants package

(PEX builds inside

docker_environment

). Setup: Pants 2.31.0 • CodeArtifact as our PyPI index, configured via

[python-repos].indexes

with token interpolation:

indexes = ["<https://aws>:%(env.CODEARTIFACT_AUTH_TOKEN)<mailto:s@our-domain.d.codeartifact.us-east-1.amazonaws.com|s@our-domain.d.codeartifact.us-east-1.amazonaws.com>/pypi/repo/simple/"]

Depot Cache (

<grpcs://cache.depot.dev>

) as REAPI remote cache • PEX binaries built in a

docker_environment

(for cross-platform Linux builds) The problem: The CodeArtifact token rotates every run (STS-based, max 12h lifetime). Since the resolved index URL (with embedded token) becomes part of PEX resolver process args, every token rotation changes the cache key for PEX resolution processes. This cascades -- different PEX output means different Docker build inputs, so

docker build

processes also miss cache. What we tried:

.netrc

approach (token outside Pants fingerprint): Works for local processes (lint/check/test) but not inside

docker_environment

-- the host

.netrc

file isn't accessible in Docker. • Writing

.netrc

into the Docker named-cache volume: The

run

goal doesn't execute in Docker environments ("The

run

goal only runs in the local environment"), so we can't trigger volume creation before needing it. •

docker_environment

mounts

field: PR #20322 was closed without merging. •

subprocess_environment_env_vars

override on `docker_environment`: Can pass env vars but can't get the

.netrc

file into the container. Current workaround: We're considering a scheduled workflow that refreshes the CodeArtifact token every 10 hours and stores it as a stable GitHub secret, so all CI runs within that window share the same token. This works but means ~2 cache busts per day. Question: Is there a better way to authenticate to a private PyPI index from inside a

docker_environment

without the credentials ending up in process cache keys? Are there plans to revisit the

mounts

feature (PR #20322) or support credential helpers / keyring for

docker_environment

processes?

Hmm, i don't think there is a solution today. We could hack something up specifically for keyring in docker, but it sounds like the robust solution would be allowing listing env vars that are available to the process but are not part of its cache key? This could potentially have other uses

Just checking that this would address the issue for you?

Root cause found:

setup-opentofu

GitHub Action modifies PATH before Pants runs We isolated the issue through binary search of our CI workflow steps. The

opentofu/setup-opentofu

action was the sole culprit. The mechanism:

setup-opentofu

calls

addPath(pathToCLI)

which prepends the OpenTofu binary directory to

PATH

via

GITHUB_PATH

. This modified

PATH

is inherited by pantsd when it starts, and Pants includes

PATH

in process cache key computation. Since the path includes runner-specific or temp-specific components, it differs between CI runs, busting every test cache key. The fix: Move

setup-opentofu

to after all Pants goals in the workflow. OpenTofu is only needed for infrastructure plan/apply steps which run after lint/check/test/package. Why lint/check/package weren't affected: We believe those goals have fewer processes whose cache keys include

PATH

, or their process construction resolves

PATH

differently. This could use further investigation

agree with Darren that it would be really nice to just ignore certain env vars for cache

So that one is still pertinent?

didn't try something like this suggestion Didn't try something like this workaround from @some-insurance-58590 yet

It was easy enough to work around (just move the file to a constant path like

/tmp/google_creds.json

& overwrite the env var). But it would be nice to have a built-in option to ignore certain env vars for cache lookups

but yeah, i think the feature request would still be helpful

so root proximate cause found 🙃