https://pantsbuild.org/ logo
#general
Title
# general
c

curved-manchester-66006

02/21/2023, 8:09 PM
I don't understand all of the moving parts in container-land, but to use pants For Real I need to be able to build docker images on gitlab runners of some sort <https://github.com/pantsbuild/pants/issues/14395>. As I understand it there are there are /many/ variants of how to build docker images besides
docker
itself that pants could use (we use Kaniko and buildkit today), there are /many/ different ways to configure gitlab runners (we can self host but the docker-in-docker options are not great for us right now) and I'm not sure what the shape of a minimal solution here looks like. Does pants grow multiple backends per tool that all interact with
docker_image
? Can each
docker_image
target have a set of tools it supports? Buildkit is sort of built in to docker, does it look simpler with that case? Sorry this is vague, my problem is that I'm unsure of both the vocabulary and the possible shape of the solutions.
c

curved-television-6568

02/21/2023, 8:27 PM
In case of supporting multiple different backend options for building Docker images I think it would make sense to have them all use
docker_image
as the “front end”.
c

curved-manchester-66006

02/21/2023, 9:38 PM
That makes sense to me. Does
docker_image
get a
backends_that_can_build_me=[]
option though? Or maybe more concretely and less pie-in-the-sky, is there an existing example where multiple backends have the same goal for the same target_type?
c

curved-television-6568

02/22/2023, 12:57 AM
do you see this as per target, rather than as a project wide option?
but that is a good question, and for the per target route, to me it seems like a field identifying a (compatible) backend would make sense, perhaps along the same lines how environments are selected.
I’m not aware of any other targets with differing backends like this.
c

curved-manchester-66006

02/22/2023, 3:20 AM
do you see this as per target, rather than as a project wide option?
I don't know! This is what I mean by not being able to see the 'shape' of the solution. If
pants.toml
had
docker.builder = {buildkit,docker}
-- bunch of hand waving here around around actually implementing that -- and you could set it to
docker
by default and
buildkit
in CI I think that would cover what I need. But: • I have no idea if that is 100, 80, or 50% of a solution for everyone else. (See the many different tools in the ticket which presumably fit different niches) • If it is plausible that different targets would only be compatible with only certain "docker builders" • How to express "docker builder" specific config in the target. Eg buildkit can do cross platform (arm,x86) builds which I don't think have a 'vanilla docker' analog. • If the number of configuration knobs for each "docker builder" turns into something like "names sets of config" where one would plausibly want to use both "buildkit-configured-this-way" and "buildkit-configured-some-other-way" in the same project. I /suspect/ most of that can reasonably be deferred after thinking out loud, but I don't really have much embodied knowledge to base my intuition on.
c

curved-television-6568

02/22/2023, 1:03 PM
I suspect builder backends would come in dedicated pants backends meaning that builder specific features could show up as specific fields for the targets using plugin fields. But thinking some more on this I have a nagging feeling we may want to tie in to environments more. Perhaps the builder should be tied (and configured) in/with a particular environment. So when building a
docker_image
the builder to use is dictated by the environment selected for that target. That will answer many questions on how to handle configuration and differences between dev & CI etc.
I’ve not yet played around with environments so are not intimately familiar with how it works yet though..
6 Views