https://pantsbuild.org/ logo
#general
Title
# general
s

stocky-raincoat-64723

02/23/2022, 4:09 PM
Hello All, I am trying to configure pants to work with remote execution, and have configured buildbarn remote execution. Although when build my package with the remote store configured I am getting the following error. Exception: InvalidArgument: "Failed to obtain offset store for instance \"main\": Unknown instance name" Can anyone direct me how to debug this issue? I think I am using the wrong remote_instance_name but I am not sure what that value should be set to.
e

enough-analyst-54434

02/23/2022, 4:21 PM
You might use this wonderful resource: https://remote-apis-testing.gitlab.io/remote-apis-testing/ You'll note the elements you care about are green in that grid. The Pants config is here: https://gitlab.com/remote-apis-testing/remote-apis-testing/-/blob/master/docker-compose-templates/docker-compose-pants.jinja2_template And build barn config here: https://gitlab.com/remote-apis-testing/remote-apis-testing/-/blob/master/docker-compose-templates/docker-compose-buildbarn.jinja2_template Hopefully that points out things you have set up differently which leads to a fix. Beyond that I'm buildbarn ignorant.
c

clean-night-52582

02/23/2022, 4:21 PM
What did you set
instanceNamePrefix
to be for the runners? Pants is looking for runners with the name
main
which probably don't exists. https://github.com/buildbarn/bb-remote-execution/blob/master/pkg/proto/remoteworker/remoteworker.proto#L54
👍 1
e

enough-analyst-54434

02/23/2022, 4:26 PM
s

stocky-raincoat-64723

02/23/2022, 4:55 PM
Thanks I've gotten a step further. Now I'm getting the exception: No workers exist for instance "remote-execution" platform {"properties": [{'name':'OSFamily', 'value': ''Linux'}]} This is the configuration I have for the workers
Copy code
{
      blobstore: common.blobstore,
      maximumMessageSizeBytes: common.maximumMessageSizeBytes,
      scheduler: { address: 'scheduler:8983' },
      httpListenAddress: common.httpListenAddress,
      maximumMemoryCachedDirectories: 1000,
      instanceName: 'remote-execution',
      browserUrl: common.browserUrl,
      buildDirectories: [{
        native: {
          buildDirectoryPath: '/worker/build',
          cacheDirectoryPath: '/worker/cache',
          maximumCacheFileCount: 10000,
          maximumCacheSizeBytes: 1024 * 1024 * 1024,
          cacheReplacementPolicy: 'LEAST_RECENTLY_USED',
        },
        runners: [{
          endpoint: { address: 'unix:///worker/runner' },
          concurrency: 8,
          platform: {
            properties: [
              { name: 'OSFamily', value: 'Linux' },
              { name: 'container-image', value: '<docker://marketplace.gcr.io/google/rbe-ubuntu16-04@sha256:b516a2d69537cb40a7c6a7d92d0008abb29fba8725243772bdaf2c83f1be2272>' },
            ],
          },
          defaultExecutionTimeout: '1800s',
          maximumExecutionTimeout: '3600s',
          workerId: {
            'pod': std.extVar("POD_NAME"),
            'node': std.extVar("NODE_NAME")
          },
        }],
      }],
    }
f

fast-nail-55400

02/23/2022, 5:07 PM
I recall that Buildbarn needs an exact match between the platform properties sent by Pants and the worker’s configured platform properties. In that config, the
container-image
property is defined on the worker and is not being sent by Pants.
👍 1
s

stocky-raincoat-64723

02/23/2022, 5:08 PM
spot on! thanks @fast-nail-55400
f

fast-nail-55400

02/23/2022, 5:09 PM
Tangentially, are you aware of the #buildbarn channel on BuildTeamWorld Slack?
1
s

stocky-raincoat-64723

02/23/2022, 5:21 PM
Are these question better to ask over there? @fast-nail-55400
Also has pants.backend.experimental.docker been tested with remote execution?
f

fast-nail-55400

02/23/2022, 5:24 PM
Not necessarily. Just mentioning it so you are aware of it since they will have deeper knowledge and experience with Buildbarn because that is where the Buildbarn developers hang out. You probably want to ask in whichever channel can answer the question. 🙂 Pants and some basic Buildbarn here, and deeper Buildbarn questions over there.
Also has pants.backend.experimental.docker been tested with remote execution?
Not to my knowledge. Also, given how the Docker cache is machine-specific, if your remote execution setup uses multiple machines, some Docker steps could be redone since the Docker cache for a specific worker might not have run an earlier build action.
and the Docker cache is entirely separate from the Buildbarn CAS (and the Pants local cache)
s

stocky-raincoat-64723

02/23/2022, 5:28 PM
for that piece I could probably configure a multi-read/write volume for the docker cache that is mounted on all the workers
9 Views