Are there ways to do cross platform docker builds in pants without using the containerd-snapshotter? We've hit a couple problems with containerd-snapshotter and are considering alternatives. Our non-pants builds are using buildx/docker-driver to do this today

Can you share an example CLI invocation that you’re currently using? That’d help work out how to map it onto a pants equivalent

I think it's essentially

docker buildx build ... --push

https://docs.docker.com/build/building/multi-platform/

Does https://www.pantsbuild.org/2.20/reference/targets/docker_image#build_platform help? Potentially something like

docker_image(..., build_platform=["linux/amd64", "linux/arm64"])

?

Here’s what I’ve tried so far: Just setting the build_platform to multiple values does not work because docker driver does not support multi-platform builds.

docker_image(
    build_platform = [
        "linux/amd64",
        "linux/arm64",
    ]
)

produces

ERROR: docker exporter does not currently support exporting manifest lists

We can force it to use a different container builder using the env var

BUILDX_BUILDER

in the pants toml. First create the builder outside of pants

docker buildx create --name builder --driver docker-container --use --bootstrap

Then set the env var in the pants toml

[docker]
use_buildx = true
env_vars = [
  "BUILDX_BUILDER=builder",
  ...
]

This will actually produce the same error as before, despite making progress

ERROR: docker exporter does not currently support exporting manifest lists

We can switch the output to

type=image

which allows

pants package

to execute successfully

docker_image(
    build_platform = [
        "linux/amd64",
        "linux/arm64",
    ],
    output={
        'type': 'image',
    },
)

But this is somewhat limited because we can’t actually push the image to a registry or consume it in another target. Trying to push the image to a registry will produce the following error

tag does not exist: <tag>

In this setup, where I want buildx to push a multi-arch manifest, the build and push commands cannot be separated. So in pants lingo, the

package

and

publish

rule need to essentially be the same, except

publish

also adds an additional cli flag

--push

Thanks for the details. Would you be interested in trying to fix 19994 along the lines of what you suggest?

yeah I’m interested in fixing it. I think 19994 is mostly addressed, except that it breaks the semantics and the defaults don’t make sense to me

Cool. The relevant code for publishing a docker image is in https://github.com/pantsbuild/pants/blob/83d7c5ed2e434cb62e4c80a7e19e3dde565b8de8/src/python/pants/backend/docker/goals/publish.py to save you the search. Ask over in #C0D7TNJHL (or let me know) if you want guidance/suggestions

What do you think a reasonable approach is here? PublishRequest usually assumes the artifact produced from

package

can be used with no modifications, in this case thats not true though. buildx has no separate “push” command like

docker push

, I need to re-invoke

docker buildx build

but with slightly different cli arguments. One way that could work is if I add and inspect the necessary fields on the publish request, then manually construct the

DockerPackageFieldSet

, and change the output type

Ahhh, hm, sorry I thought the publish request explicitly invoked the package request, but reading more closely I see that's not the case. I think you'll need to ask for someone more expert than me in #C0D7TNJHL 😄

no worries, thanks for the help so far