I am noticing that in CI caching

~/.cache/pants/setup

doesn't cache any external pants plugins (global.plugins) and those get downloaded on every CI run. any idea which directory should be cached in order to avoid the re-download in every CI run. I found no reference to that in https://www.pantsbuild.org/docs/using-pants-in-ci (note that we use remote caching, so we nuke lmdb_store and named_caches before uploading the data to the CI cache)

I think you won't like the answer. By the point we're loading plugins, we're dogfooding the engine; so normal Python backend rules are used. As such, the plugins are mixed in with everything else we download for linting, typechecking, tests, etc. Namely, these are all stored in the local LMDB cache ~/.cache/pants/lmdb_store and they are exploded into a venv in ~/.cache/pants/named_caches (In particular ~/.cache/pants/named_caches/pex_root).

so will they come from a remote cache if there is one ? or downloaded and installed from pypi ?

That's actually likely. We use the engine, but not with all options necessarily wired up since part of collecting all options is gathering plugins and then probing them. That said, caching options are all core GLOBAL options so this need not be the case. Handwaving from my phone, but the nexus of code is here to peer at: + https://github.com/pantsbuild/pants/blob/main/src/python/pants/init/plugin_resolver.py + https://github.com/pantsbuild/pants/blob/main/src/python/pants/init/options_initializer.py

ok. will submit an issue on Monday.

at the point when we are downloading the plugin, we can’t possibly use the plugin to auth to a remote cache =/

I would expect the plugins to be stored/installed into the virtualenv where pants is installed (under ~/.cache/pants/setup)

it will, but in our standard cache location:

~/.cache/pants/named_caches

yeah... not a good candidate for CI caching since it gets pretty big.