https://pantsbuild.org/ logo
Join Slack
Powered by
can I provide the buildkit `--cache-from` argument...
# general
w
can I provide the buildkit 
--cache-from
argument to the 
docker_image
via some pants invocation?
It does not look like it has a corresponding environment variable to set in 
--docker-env-vars
s
not with “vanilla” pants, but it’s possible to add via plugin in your repo
✔️ 1
AFAICT it’s not possible to tell Pants to run 
docker buildx build
instead of 
docker build
, unless you write an entirely new plugin to replace the built-in docker support. we are working around this by running 
docker buildx install
everywhere as part of our standard setup process
w
we're only utilizing the feature subset exposed by 
DOCKER_BUILDKIT=1
, so was hoping for some luck with the 
--cache-from
arg
👍 1
s
this is our 
register.py
, you can copy-paste and delete the stuff for `--cache-to`:
Copy code
from typing import Iterator

from pants.backend.docker.target_types import (
    DockerBuildOptionFieldMixin,
    DockerImageTarget,
    OptionValueFormatter,
)
from pants.engine.target import StringField
from pants.util.strutil import softwrap


class DockerImageCacheFromField(DockerBuildOptionFieldMixin, StringField):
    alias = "cache_from"
    help = softwrap(
        """
        External cache sources (e.g., "user/app:cache", "type=local,src=path/to/dir")

        NOTE: To use this field, you must run `docker buildx install` on your host.
        """
    )
    docker_build_option = "--cache-from"

    def option_values(self, *, value_formatter: OptionValueFormatter) -> Iterator[str]:
        if self.value:
            yield value_formatter(self.value)


class DockerImageCacheToField(DockerBuildOptionFieldMixin, StringField):
    alias = "cache_to"
    help = softwrap(
        """
        Cache export destinations (e.g., "user/app:cache", "type=local,dest=path/to/dir")

        NOTE: To use this field, you must run `docker buildx install` on your host.
        """
    )
    docker_build_option = "--cache-to"

    def option_values(self, *, value_formatter: OptionValueFormatter) -> Iterator[str]:
        if self.value:
            yield value_formatter(self.value)


def rules():
    return [
        DockerImageTarget.register_plugin_field(DockerImageCacheFromField),
        DockerImageTarget.register_plugin_field(DockerImageCacheToField),
    ]
❤️ 2
w
Awesome, saved me some grokk 🙏
🙌 1
c
Wow, this is so awesome. Seeing how easy it was to fix the shortcomings in the docker backend for this! Thx for sharing @sparse-lifeguard-95737 👍
h
Thank you Dan for sharing this! @curved-television-6568 thoughts on the best way to upstream that?
c
Could take it pretty much as is, only thing to would be to maybe put it behind some sort of feature flag whether to use buildkit or not. (and when 
use_buildkit=true
we could set the 
DOCKER_BUILDKIT=1
env too)
s
if it was possible to make the 
docker build
command pluggable you could possibly have a separate plugin that replaces 
build
with 
buildx build
, and adds support for all the buildx-specific fields
c
should just finish the ticket adding proper buildx support to the docker backend 😛 (also making it pluggable wouldn’t be a bad idea, of course)
6 Views
Open in Slack
PreviousNext
Powered by