https://pantsbuild.org/ logo
#general
Title
# general
t

thousands-plumber-33255

08/08/2022, 11:06 AM
Hi all šŸ™‚ I am trying to use remote caching (not execution) using https://www.buildbuddy.io/. I know that is not listed here, but it is here why I was assuming that it might work with pants as well. It has a good free plan and good docs for configuring a deployment to the cloud. The caching sections looks as follows:
Copy code
local_cache = false
remote_cache_read = true
remote_cache_write = true
remote_store_address = "<grpcs://remote.buildbuddy.io>"
remote_store_headers = "{'user-agent': 'pants', 'x-buildbuddy-api-key': 'token'}"
remote_instance_name = "buildbuddy-io/buildbuddy/local"
remote_store_chunk_upload_timeout_seconds="3600"
I could not confirm that it worked to be honest. Can I somehow test the remote connection or enable verbose logs?
Copy code
vscode āžœ $ ./pants test libs/lib_common/tests/grid_test.py 
10:33:33.20 [INFO] Initializing scheduler...
10:33:33.48 [INFO] Scheduler initialized.
10:33:35.11 [INFO] Canceled: Building pytest.pex from lockfiles/python/tools/pytest
10:33:35.41 [INFO] Canceled: Building pytest.pex from lockfiles/python/tools/pytest
10:33:36.68 [INFO] Completed: Building 3 requirements for requirements.pex from the lockfiles/python/default resolve: Shapely==1.7.1, numpy==1.21.0, pyproj==3.1.0
10:33:37.35 [INFO] Completed: Building local_dists.pex
10:33:37.81 [INFO] Canceled: Building pytest_runner.pex
10:33:38.28 [INFO] Completed: Run Pytest - libs/lib_common/tests/grid_test.py succeeded.

āœ“ libs/lib_common/tests/grid_test.py succeeded in 0.90s (cached remotely).
vscode āžœ $ ./pants test libs/lib_common/tests/grid_test.py 
10:34:06.47 [INFO] Completed: Run Pytest - libs/lib_common/tests/grid_test.py succeeded.

āœ“ libs/lib_common/tests/grid_test.py succeeded in 0.90s (memoized).
The pants logs seem okay, but I cannot see any artifacts in the buildbuddy UI. Did I miss something?
b

bitter-ability-32190

08/08/2022, 12:29 PM
The "memoized" means the daemon had the result cached. If you were pulling from the remote cache you'd see "cached remotely". You can pass
--stats-log
(I think) to see the stats at the end of the command (includes caching info). Also
--level=debug
might help
t

thousands-plumber-33255

08/08/2022, 12:45 PM
When running a test for the first time,
--stats-log
indicates that cache was written successfully:
Copy code
12:36:19.30 [INFO] Counters:
  backtrack_attempts: 0
  local_cache_read_errors: 0
  local_cache_requests: 0
  local_cache_requests_cached: 0
  local_cache_requests_uncached: 0
  local_cache_total_time_saved_ms: 0
  local_cache_write_errors: 0
  local_execution_requests: 17
  local_process_total_time_run_ms: 13297
  remote_cache_read_errors: 0
  remote_cache_requests: 16
  remote_cache_requests_cached: 2
  remote_cache_requests_uncached: 4
  remote_cache_speculation_local_completed_first: 10
  remote_cache_speculation_remote_completed_first: 2
  remote_cache_total_time_saved_ms: 883
  remote_cache_write_attempts: 14
  remote_cache_write_errors: 0
  remote_cache_write_successes: 13
  remote_execution_errors: 0
  remote_execution_requests: 0
  remote_execution_rpc_errors: 0
  remote_execution_rpc_execute: 0
  remote_execution_rpc_retries: 0
  remote_execution_rpc_wait_execution: 0
  remote_execution_success: 0
  remote_execution_timeouts: 0
  remote_process_total_time_run_ms: 0
  remote_store_missing_digest: 0
But how can I confirm a read works for a subsequent run? I was expecting that the result cached by the daemon would not be used as I have set
local_cache = false
.
I added the
--no-pantsd
flag and got this returned:
Copy code
āœ“ libs/lib_common/tests/aws_test.py succeeded in 3.82s (cached remotely).
12:48:04.60 [DEBUG] Completed: `test` goal
12:48:04.60 [DEBUG] computed 1 nodes in 8.870665 seconds. there are 892 total nodes.
12:48:04.60 [DEBUG] Async completion is disabled: waiting for workunit callbacks to complete...
12:48:04.60 [INFO] Counters:
  backtrack_attempts: 0
  local_cache_read_errors: 0
  local_cache_requests: 0
  local_cache_requests_cached: 0
  local_cache_requests_uncached: 0
  local_cache_total_time_saved_ms: 0
  local_cache_write_errors: 0
  local_execution_requests: 17
  local_process_total_time_run_ms: 2340
  remote_cache_read_errors: 0
  remote_cache_requests: 16
  remote_cache_requests_cached: 5
  remote_cache_requests_uncached: 1
  remote_cache_speculation_local_completed_first: 10
  remote_cache_speculation_remote_completed_first: 5
  remote_cache_total_time_saved_ms: 6621
  remote_cache_write_attempts: 11
  remote_cache_write_errors: 0
  remote_cache_write_successes: 11
  remote_execution_errors: 0
  remote_execution_requests: 0
  remote_execution_rpc_errors: 0
  remote_execution_rpc_execute: 0
  remote_execution_rpc_retries: 0
  remote_execution_rpc_wait_execution: 0
  remote_execution_success: 0
  remote_execution_timeouts: 0
  remote_process_total_time_run_ms: 0
  remote_store_missing_digest: 0
Does it look right? If so, I will need to figure out why I am not seeing any results in the BuildBuddy UI.
b

bitter-ability-32190

08/08/2022, 2:36 PM
Copy code
remote_cache_requests_cached: 5
Yeah looks right to me. BuildBuddy might be lagging? No idea šŸ™‚ Someone with BuildBuddy experience might have to chime in
āœ… 1
f

fresh-cat-90827

04/10/2023, 11:42 AM
As of April 2023, the configuration above does with the free tier of BuildBuddy. After running
./pants --no-pantsd test ::
, you should be able to see the action cache hits under https://app.buildbuddy.io/usage/. Do note that it may take a few minutes before the usage is updated after you ran the command.