Hi all 🙂 I am trying to use remote caching (not execution) using https://www.buildbuddy.io/. I know that is not listed here, but it is here why I was assuming that it might work with pants as well. It has a good free plan and good docs for configuring a deployment to the cloud. The caching sections looks as follows:

local_cache = false
remote_cache_read = true
remote_cache_write = true
remote_store_address = "<grpcs://remote.buildbuddy.io>"
remote_store_headers = "{'user-agent': 'pants', 'x-buildbuddy-api-key': 'token'}"
remote_instance_name = "buildbuddy-io/buildbuddy/local"
remote_store_chunk_upload_timeout_seconds="3600"

I could not confirm that it worked to be honest. Can I somehow test the remote connection or enable verbose logs?

vscode ➜ $ ./pants test libs/lib_common/tests/grid_test.py 
10:33:33.20 [INFO] Initializing scheduler...
10:33:33.48 [INFO] Scheduler initialized.
10:33:35.11 [INFO] Canceled: Building pytest.pex from lockfiles/python/tools/pytest
10:33:35.41 [INFO] Canceled: Building pytest.pex from lockfiles/python/tools/pytest
10:33:36.68 [INFO] Completed: Building 3 requirements for requirements.pex from the lockfiles/python/default resolve: Shapely==1.7.1, numpy==1.21.0, pyproj==3.1.0
10:33:37.35 [INFO] Completed: Building local_dists.pex
10:33:37.81 [INFO] Canceled: Building pytest_runner.pex
10:33:38.28 [INFO] Completed: Run Pytest - libs/lib_common/tests/grid_test.py succeeded.

✓ libs/lib_common/tests/grid_test.py succeeded in 0.90s (cached remotely).
vscode ➜ $ ./pants test libs/lib_common/tests/grid_test.py 
10:34:06.47 [INFO] Completed: Run Pytest - libs/lib_common/tests/grid_test.py succeeded.

✓ libs/lib_common/tests/grid_test.py succeeded in 0.90s (memoized).

The pants logs seem okay, but I cannot see any artifacts in the buildbuddy UI. Did I miss something?

The "memoized" means the daemon had the result cached. If you were pulling from the remote cache you'd see "cached remotely". You can pass

--stats-log

(I think) to see the stats at the end of the command (includes caching info). Also

--level=debug

might help

When running a test for the first time,

--stats-log

indicates that cache was written successfully:

12:36:19.30 [INFO] Counters:
  backtrack_attempts: 0
  local_cache_read_errors: 0
  local_cache_requests: 0
  local_cache_requests_cached: 0
  local_cache_requests_uncached: 0
  local_cache_total_time_saved_ms: 0
  local_cache_write_errors: 0
  local_execution_requests: 17
  local_process_total_time_run_ms: 13297
  remote_cache_read_errors: 0
  remote_cache_requests: 16
  remote_cache_requests_cached: 2
  remote_cache_requests_uncached: 4
  remote_cache_speculation_local_completed_first: 10
  remote_cache_speculation_remote_completed_first: 2
  remote_cache_total_time_saved_ms: 883
  remote_cache_write_attempts: 14
  remote_cache_write_errors: 0
  remote_cache_write_successes: 13
  remote_execution_errors: 0
  remote_execution_requests: 0
  remote_execution_rpc_errors: 0
  remote_execution_rpc_execute: 0
  remote_execution_rpc_retries: 0
  remote_execution_rpc_wait_execution: 0
  remote_execution_success: 0
  remote_execution_timeouts: 0
  remote_process_total_time_run_ms: 0
  remote_store_missing_digest: 0

But how can I confirm a read works for a subsequent run? I was expecting that the result cached by the daemon would not be used as I have set

local_cache = false

.

Copy code

remote_cache_requests_cached: 5

Yeah looks right to me. BuildBuddy might be lagging? No idea 🙂 Someone with BuildBuddy experience might have to chime in

As of April 2023, the configuration above does with the free tier of BuildBuddy. After running

./pants --no-pantsd test ::

, you should be able to see the action cache hits under https://app.buildbuddy.io/usage/. Do note that it may take a few minutes before the usage is updated after you ran the command.