https://pantsbuild.org/ logo
Powered by
w

worried-painter-31382

09/02/2022, 1:57 PM
can I provide the buildkit 
--cache-from
argument to the 
docker_image
via some pants invocation?
It does not look like it has a corresponding environment variable to set in 
--docker-env-vars
s

sparse-lifeguard-95737

09/02/2022, 2:03 PM
not with “vanilla” pants, but it’s possible to add via plugin in your repo
✔️ 1
AFAICT it’s not possible to tell Pants to run 
docker buildx build
instead of 
docker build
, unless you write an entirely new plugin to replace the built-in docker support. we are working around this by running 
docker buildx install
everywhere as part of our standard setup process
w

worried-painter-31382

09/02/2022, 2:06 PM
we're only utilizing the feature subset exposed by 
DOCKER_BUILDKIT=1
, so was hoping for some luck with the 
--cache-from
arg
👍 1
s

sparse-lifeguard-95737

09/02/2022, 2:06 PM
this is our 
register.py
, you can copy-paste and delete the stuff for `--cache-to`:
Copy code
from typing import Iterator

from pants.backend.docker.target_types import (
    DockerBuildOptionFieldMixin,
    DockerImageTarget,
    OptionValueFormatter,
)
from pants.engine.target import StringField
from pants.util.strutil import softwrap


class DockerImageCacheFromField(DockerBuildOptionFieldMixin, StringField):
    alias = "cache_from"
    help = softwrap(
        """
        External cache sources (e.g., "user/app:cache", "type=local,src=path/to/dir")

        NOTE: To use this field, you must run `docker buildx install` on your host.
        """
    )
    docker_build_option = "--cache-from"

    def option_values(self, *, value_formatter: OptionValueFormatter) -> Iterator[str]:
        if self.value:
            yield value_formatter(self.value)


class DockerImageCacheToField(DockerBuildOptionFieldMixin, StringField):
    alias = "cache_to"
    help = softwrap(
        """
        Cache export destinations (e.g., "user/app:cache", "type=local,dest=path/to/dir")

        NOTE: To use this field, you must run `docker buildx install` on your host.
        """
    )
    docker_build_option = "--cache-to"

    def option_values(self, *, value_formatter: OptionValueFormatter) -> Iterator[str]:
        if self.value:
            yield value_formatter(self.value)


def rules():
    return [
        DockerImageTarget.register_plugin_field(DockerImageCacheFromField),
        DockerImageTarget.register_plugin_field(DockerImageCacheToField),
    ]
❤️ 2
w

worried-painter-31382

09/02/2022, 2:09 PM
Awesome, saved me some grokk 🙏
🙌 1
c

curved-television-6568

09/02/2022, 2:38 PM
Wow, this is so awesome. Seeing how easy it was to fix the shortcomings in the docker backend for this! Thx for sharing @sparse-lifeguard-95737 👍
h

hundreds-father-404

09/02/2022, 3:35 PM
Thank you Dan for sharing this! @curved-television-6568 thoughts on the best way to upstream that?
c

curved-television-6568

09/02/2022, 3:37 PM
Could take it pretty much as is, only thing to would be to maybe put it behind some sort of feature flag whether to use buildkit or not. (and when 
use_buildkit=true
we could set the 
DOCKER_BUILDKIT=1
env too)
s

sparse-lifeguard-95737

09/02/2022, 3:39 PM
if it was possible to make the 
docker build
command pluggable you could possibly have a separate plugin that replaces 
build
with 
buildx build
, and adds support for all the buildx-specific fields
c

curved-television-6568

09/02/2022, 3:45 PM
should just finish the ticket adding proper buildx support to the docker backend 😛 (also making it pluggable wouldn’t be a bad idea, of course)
6 Views
Powered by