https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-breakfast-49010

02/12/2020, 9:17 PM
I'm currently looking into the problem of how to gracefully handle errors from people with macs trying to run goals remotely on linux build machines (and eventually mkae this work without erroring), and I realize I have a lot of gaps in my knowledge about how remoting actually works
for instance, how much of remoting relies on specific details of the API for whatever cloud service we're using? does pants encode anything specific about gcloud, for instance?
w

witty-crayon-22786

02/12/2020, 10:01 PM
almost nothing. we have a nightly job that validates that it works against Scoot: another backend
@early-needle-54791 did some prior work here: see https://github.com/pantsbuild/pants/issues/8090
and the linked tickets
e

early-needle-54791

02/12/2020, 10:03 PM
we discussed a bit in DMs
w

witty-crayon-22786

02/12/2020, 10:03 PM
👍
h

hundreds-breakfast-49010

02/12/2020, 10:05 PM
so, when a local user running pants kicks off a remote job, how do cached artifacts actually get from the pants process running locally to the one running remotely?
or vice-versa?
w

witty-crayon-22786

02/12/2020, 10:05 PM
the rust
store
crate has local and remote implementations
before executing a remote process, inputs are synced local to remote if they don't already exist.
h

hundreds-breakfast-49010

02/12/2020, 10:11 PM
so, before that happens, does the local pants start a pants process remotely?
or does remoting only work if there's already a pants process running on the remote machine, and listening for remote requests somehow?
w

witty-crayon-22786

02/12/2020, 10:13 PM
e

early-needle-54791

02/12/2020, 10:13 PM
The remoting aspect sends an RPC to the remote host, which runs a bare process. The remote isn’t necessarily running pants.
though I suppose it could, for example to remote integration tests.
w

witty-crayon-22786

02/12/2020, 10:13 PM
from an external perspective, whether there are long lived workers is not exposed... each
Execute
call is completely independent
h

hundreds-breakfast-49010

02/12/2020, 10:23 PM
so the bazel api is a well-known api that gcloud supports, and that pants knows how to use to run remote executions?
w

witty-crayon-22786

02/12/2020, 10:23 PM
yep
h

hundreds-breakfast-49010

02/12/2020, 10:24 PM
(and that logic lives in
store
I suppose)
w

witty-crayon-22786

02/12/2020, 10:24 PM
it is split across
store
and
process_execution
which both have remote halves
h

hundreds-breakfast-49010

02/12/2020, 10:24 PM
makes sense i've seen some of that code before
I always wondered why the file system code was using a bunch of functions namedspaced under
bazel
a

aloof-angle-91616

02/13/2020, 9:59 PM
@hundreds-breakfast-49010 i would love an excuse to get https://github.com/pantsbuild/pants/pull/7553 in too
an integration test that validates that we can set up and tear down a scoot cluster to perform remote execution against (running locally)
h

hundreds-breakfast-49010

02/13/2020, 9:59 PM
that definitely seems like a useful thing ot have at hand
a

aloof-angle-91616

02/13/2020, 10:00 PM
i don't expect it will increase CI time and the blockers to merging it are well understood
2 Views