Anyone know offhand if pants supports zstd compres...
# generalf
fresh-mechanic-68429
04/25/2025, 8:24 PMAnyone know offhand if pants supports zstd compression when using a remote cache? From a quick look at byte_store.rs I don't think it does, but wanted to check before opening an issue.
I believe, but am not totally sure, that this feature would allow me to offload zstd compression from the remote server to the pants client, reducing network transfer/latency.
This is the api in question https://github.com/bazelbuild/remote-apis/pull/168 which I think is fully supported by bazel-remote
fresh-mechanic-68429
04/25/2025, 8:27 PMadditional context, we're seeing some pretty long
check_action_cachef
fast-nail-55400
04/25/2025, 8:37 PMTonic (the Rust gRPC crate) does seem to support it, but it would need to be configured. https://github.com/hyperium/tonic/pull/1532
f
fresh-mechanic-68429
04/25/2025, 9:02 PMI think this needs to be implemented outside of grpc. The high level idea is that the remote cache stores the blobs in zstd on disk and serves them back directly, without additional round of decompression/compression
This is explicitly called out in the api
When downloading compressed blobs:
...
Clients SHOULD NOT use gRPC-level compression, since this would compress already-compressed data.
f
fast-nail-55400
04/25/2025, 9:04 PMah yeah, I forgot that the REAPI has compression defined in the protocol.
fast-nail-55400
04/25/2025, 9:04 PMfor that, we would need to modify the pants REAPI client to implement support
👍 1
f
fresh-mechanic-68429
04/25/2025, 9:08 PMok I'll open an issue to track it, I might be able to put some effort into this
fresh-mechanic-68429
04/25/2025, 9:15 PM14 Views