I could use some help debugging the remote caching server. I have a local instance of the buildfarm build-server running off of this documentation and my pants.toml configured like:

remote_cache_read = true
remote_cache_write = true
remote_store_address = "<grpc://localhost:8980>"
remote_instance_name = "memory"

I could use help figuring out if the pants build actually hits the running remote cache instance.

If you don't get errors at the top of your goals, then in my experience, it's hitting it.

Thanks that looks like it is working then, couldn't believe it myself. So say I'm using the remote cache and building a docker image, does that mean there's a copy of the built docker image in the remote cache instance?

I would assume so. I didn't see anything for configuring what does or doesn't hit the cache. But all of these systems clear the cache based on least recently used basis so that would take up a couple hundred MB (or a few GB depending on image) at one time and then probably quickly get pushed out of the cache from tests and other things that get run/accessed more frequently.

add

--stats-log

to your Pants invocation to view the internal counters Pants has

Thanks that's very helpful. I'm going to play with the remote server a bit because it looks like its not storing the docker image artifacts. Though it is getting hit.

remote_cache_requests: 4
  remote_cache_requests_cached: 0
  remote_cache_requests_uncached: 4

As you mentioned @high-yak-85899 its probably getting evicted due to its size. My end goal is to prototype a configuration where the client does not have the docker images in its local cache.

Reliably storing container images should be left to something like a container registry imo.

@high-yak-85899 that sounds right, gonna read some more documentation and think about it.

The remote cache key is the hash of the encoded protobuf form of the

Process

executed by Pants. (The protobufs involved are the

Action

and

Command

protos from the Remote Execution API.) Enable debug logging with

-ldebug

and see if the

Process

invocation by Pants have some difference which would vary their content from run to run.

Further, building a Docker image intentionally isn't cached: https://github.com/pantsbuild/pants/pull/13464 I recommend using something like

./pants lint

or

./pants test

when debugging remote caching

But the pex binaries that would be copied into an image would be cached, right?

Correct