https://pantsbuild.org/ logo
Join Slack
Powered by
I'm currently looking into the problem of how to g...
# development
h
I'm currently looking into the problem of how to gracefully handle errors from people with macs trying to run goals remotely on linux build machines (and eventually mkae this work without erroring), and I realize I have a lot of gaps in my knowledge about how remoting actually works
for instance, how much of remoting relies on specific details of the API for whatever cloud service we're using? does pants encode anything specific about gcloud, for instance?
w
almost nothing. we have a nightly job that validates that it works against Scoot: another backend
@early-needle-54791 did some prior work here: see https://github.com/pantsbuild/pants/issues/8090
and the linked tickets
e
we discussed a bit in DMs
w
👍
h
so, when a local user running pants kicks off a remote job, how do cached artifacts actually get from the pants process running locally to the one running remotely?
or vice-versa?
w
the rust 
store
crate has local and remote implementations
before executing a remote process, inputs are synced local to remote if they don't already exist.
h
so, before that happens, does the local pants start a pants process remotely?
or does remoting only work if there's already a pants process running on the remote machine, and listening for remote requests somehow?
w
e
The remoting aspect sends an RPC to the remote host, which runs a bare process. The remote isn’t necessarily running pants.
though I suppose it could, for example to remote integration tests.
w
from an external perspective, whether there are long lived workers is not exposed... each 
Execute
call is completely independent
h
so the bazel api is a well-known api that gcloud supports, and that pants knows how to use to run remote executions?
w
yep
h
(and that logic lives in 
store
I suppose)
w
it is split across 
store
and 
process_execution
which both have remote halves
h
makes sense i've seen some of that code before
I always wondered why the file system code was using a bunch of functions namedspaced under 
bazel
a
@hundreds-breakfast-49010 i would love an excuse to get https://github.com/pantsbuild/pants/pull/7553 in too
an integration test that validates that we can set up and tear down a scoot cluster to perform remote execution against (running locally)
h
that definitely seems like a useful thing ot have at hand
a
i don't expect it will increase CI time and the blockers to merging it are well understood
2 Views
Open in Slack
PreviousNext
Powered by