https://pantsbuild.org/ logo
c

clever-father-91273

05/16/2023, 10:22 AM
Hello! I'm having some trouble with remote caching. After reading the Pants documentation and browsing the internet, I'm under the impression that to enable remote caching of build results, one has to set up a gRPC solution. Such solutions allow for not only remote caching, but also remote execution, which I don't need (I'm running the builds in my GCP environment). It feels strange to have to set up and maintain a complex solution only to use a small part of its capability (the caching). Is it in some way possible to simply use a file system for storing the cache? In Bazel, for instance, one can simply pass a GCP bucket as a location for storing the cache. Would a similar solution be possible with Pants? Thanks!
h

high-magician-46188

05/16/2023, 10:37 AM
This sounds to me like a really slow cache. All it requires is spinning up a container that implements the cache part of the REAPI protocal, it's pretty simple IMHO. At my workplace, we use https://github.com/buchgr/bazel-remote. Disclaimer: I'm not a maintainer.
c

clever-father-91273

05/16/2023, 10:39 AM
Right, but what we don't like is the need to maintain a solution capable of remote execution while not actually using it.
h

high-magician-46188

05/16/2023, 10:40 AM
Well, this one only caches, it doesn't execute.
b

broad-processor-92400

05/16/2023, 11:08 AM
There’s not one built-in to pants, but it would be nice (covered by https://github.com/pantsbuild/pants/issues/11149). I understand that it’s theoretically possible to run bazel-remote locally but backed by cloud S3/GCP bucket. I think this could be run in the background within each CI executor with a
<gRPC://localhost>:…
url, as, effectively, translator from local gRPC to/from remote blob store. (I’ve got half way through trying this but not yet finished it, so can’t speak personally)
b

bitter-ability-32190

05/16/2023, 11:08 AM
Yeah we use and love bazel-remote as well. The docker image they provide is plug-and-play
b

broad-processor-92400

05/16/2023, 11:09 AM
(This saves running the high-power/surface area remote execution server accessible over the network)
b

bitter-ability-32190

05/16/2023, 11:10 AM
Also seems weird to not want to user something because it could do something else. Pants itself supports a bunch of languages/tools that I don't anyone is using them all 🙃
c

curved-manchester-66006

05/16/2023, 1:59 PM
Should
bazel-remote
be listed as an option at https://www.pantsbuild.org/docs/remote-caching-execution ?
b

bitter-ability-32190

05/16/2023, 3:12 PM
c

curved-manchester-66006

05/16/2023, 3:20 PM
Well, I didn't know this project existed before today. It looks promising! But I'm not sure I can attest to "Pants is known to work with" it
b

bitter-ability-32190

05/16/2023, 3:22 PM
I can and am happy to approve the PR 🙂
(We like making others feel empowered to open issues and PRs 😉)
3 Views