Hey everyone, I hope you're all doing well! So, I've been experimenting with the remote caching capabilities of Pants, and it looks great! One thing that I am a bit worried about concurrency, though. I am running Pants through many CI steps that run concurrently. I have

concurrent = true

set in my

pants.toml

file and things seems to be working fine. Now that I enabled remote caching, is there any issues that may happen by running pants in parallel while reading and writing to a remote cache? Is Pants smart enough to not have issues with this use case? Thanks for the help! 😊 Oh, btw, I am running the gRPC service as a sidecar in my CI pipeline and recording the cached data in S3.

Running concurrently for caching should be fine for pants. Does bazel-remote expect to be running just one instance? For your concurrency, is it on a single machine or is it across different machines? (Eg if you’re using GitHub actions, is it all within the execution of a single “job”, or seperate jobs (including a matrix).)

Hey Huon, sorry for taking so long to answer. We're running pants inside a CI job using Drone. I think all steps runs in the same machine. I just wanted to be sure there are nothing very obvious that could fail when running multiple pants commands while hitting the same bazel-remote service. Thank you for your answer! 😄

We're running pants inside a CI job using Drone. I think all steps runs in the same machine.

Ah, okay. I think I asked an irrelevant question about physical machines, a better one is: are all the steps running in the same check-out/same directory? I believe that's the only time

concurrent = true

is required, if they're separate directories, I think they should be able to run without that, and thus be slightly faster (

concurrent = true

disables the pantsd daemon, and thus requires doing more work on each invocation instead of letting it be cached by the background daemon).

Hey Huon, thank you again for the answer. Currently, we're running Pants steps in the same directory, but it's good to know that it's not necessary to set

concurrent = true

when running pants in different directories even if we're running many instances of pants at the same time.