This is a Gitlab CI environment where I'm having issues. There's no kubeconfig on the CI worker node The relevant pants.toml docker backend config is

[docker]
use_buildx = true
build_verbose = true
env_vars = [
    "DOCKER_CONFIG=%(homedir)s/.docker",
    "DOCKER_HOST",
    "BUILDX_BUILDER=ci",
    "BUILDKIT_NAMESPACE=buildkit-platform",
    "PATH",
]
...

When I run

docker buildx ls

, I get an output very similar to this:

docker buildx ls
NAME/NODE                     DRIVER/ENDPOINT                                                       STATUS    BUILDKIT   PLATFORMS
ci*                           kubernetes                                                                                 
 \_ buildkit-platform-amd64    \_ kubernetes:///ci?deployment=buildkit-platform-amd64&kubeconfig=   running   v0.21.1    linux/amd64 (+4), linux/386
 \_ buildkit-platform-arm64    \_ kubernetes:///ci?deployment=buildkit-platform-arm64&kubeconfig=   running   v0.21.1    linux/arm64, linux/arm (+2)
default                                                                                             error

I'm trying to use that "ci" buildx driver, hence the

BUILDX_BUILDER=ci

according to

-ldebug

output, pants is running

/usr/bin/docker buildx build \
--platform=linux/amd64 \
--label target=prod \
--output=type=docker \
--pull=False \
--tag <http://registry.ddbuild.io/ci/dd-analytics/run_squares:v1016161113-848c0969|registry.ddbuild.io/ci/dd-analytics/run_squares:v1016161113-848c0969> \
--build-arg CI_COMMIT_REF_SLUG \
--build-arg CI_COMMIT_SHORT_SHA \
--build-arg CI_JOB_ID \
--file subprojects/example/src/python/example/app/ray_job/Dockerfile.run_squares \
.

One more bit of info: the

~/.docker/buildx/instances/ci

file is present and has this content

{
  "Name": "ci",
  "Driver": "kubernetes",
  "Nodes": [
    {
      "Name": "buildkit-platform-amd64",
      "Endpoint": "kubernetes:///ci?deployment=buildkit-platform-amd64&kubeconfig=",
      "Platforms": [],
      "DriverOpts": {
        "namespace": "buildkit-platform"
      },
      "Flags": [
        "--allow-insecure-entitlement=network.host"
      ],
      "Files": null
    },
    {
      "Name": "buildkit-platform-arm64",
      "Endpoint": "kubernetes:///ci?deployment=buildkit-platform-arm64&kubeconfig=",
      "Platforms": null,
      "DriverOpts": {
        "namespace": "buildkit-platform"
      },
      "Flags": [
        "--allow-insecure-entitlement=network.host"
      ],
      "Files": null
    }
  ],
  "Dynamic": false
}

I assumed because we have

"DOCKER_CONFIG=%(homedir)s/.docker"

set in pants.toml that docker/buildx would 'see' this config

@happy-kitchen-89482 @nutritious-hair-72580 sorry to ping you (I tried to look at the git history to see who's worked in this area)... I was wondering if you had any insight or threads to pull on my end. Or if you know who might have ideas Thanks in advance 🙏

@nutritious-hair-72580 I'm also have doubts that passing

--builder=kube

alone will be enough to get this working. I set

BUILDX_BUILDER=ci

, and

BUILDX_BUILDER

is supposed to do the same thing as the

--builder

argument (see https://docs.docker.com/reference/cli/docker/buildx/#builder)

That could be fixed relatively easily

I got it working! If anyone else comes searching for how to get

pants package

to build a

docker_image

target with buildx using a kubernetes driver type: • I set

BUILDX_BUILDER=<my buildx driver name>

in [docker].envvars to configure the buildx driver ➡️ I wrote up a feature request to make this configuration more natural for pantsbuild https://github.com/pantsbuild/pants/issues/22487 • Also in [docker].envvars, I had to configure pants to be aware of

KUBERNETES_SERVICE_[HOST|PORT|PORT_HTTPS]

environment variables. Because pants is running "in-cluster," buildx needs those env vars to correctly instantiate the config client that will pull config from the kubernetes cluster it's in • Because I'm using a kuberenetes buildx driver and I don't have a docker daemon running, I had to set the output of my

docker_image

target to

type=registry

. This also means I had to stop doing

pants publish

... the

pants package

step does the publishing thanks to this output type. What's awesome is that there was a way to this pants package of a docker image to work with pants as it is (I'm using pants v2.27.0). At a minimum, some documentation about this case would be helpful to add to the pants docs.