agreeable-oyster-28981
06/08/2022, 9:50 PMsquare-forest-55300
06/08/2022, 10:10 PMagreeable-oyster-28981
06/08/2022, 10:11 PMsquare-forest-55300
06/08/2022, 10:33 PMfaint-businessperson-86903
06/08/2022, 10:36 PMapt-get install -y <http://docker.io|docker.io>
)
3. Add the following mount so that the builder container can access the same docker socket as the host: /var/run/docker.sock:/var/run/docker.sock
square-forest-55300
06/08/2022, 10:38 PMfaint-businessperson-86903
06/08/2022, 10:39 PMsquare-forest-55300
06/08/2022, 10:41 PMfaint-businessperson-86903
06/08/2022, 10:41 PM~/.cache/pants
as wellsquare-forest-55300
06/08/2022, 10:43 PMfaint-businessperson-86903
06/08/2022, 10:43 PMsquare-forest-55300
06/08/2022, 10:44 PMgitlab/gitlab-runner:latest
agreeable-oyster-28981
06/09/2022, 12:21 PMto add on top, I think i remember that Gitlab has some form of distributed artifact cache. I feel very uncomfortable thinking about running pants like this. I would use it as a dedicated build server.Not sure what you mean by distributed. In our case, we’re using s3 as a storage for artefact caching. Which is not “distributed” (as, there is a single source of truth), however, the runners are distributed yes, using a single cache source
i think those images get trashed after finishing, right?
correct me, if i am wrong. You want to run docker in docker to run a gitlab runner that handles pants. Once that run finished, the image gets trashed and cleaned up.Yes, they’re short time living container, once the job is finished, selected artifacts are cache, the rest is discarded.
Although we’re not currently using the pants docker integration for building the ‘inner’ image@faint-businessperson-86903 I’m curious, what’s the reason for that? Currently for my testing, I’m also not using the
docker_image
target, because of the lack of multi platform support (M1/arm64, I need to build an amd64 image). Is there any other limitations?square-forest-55300
06/09/2022, 12:47 PMfaint-businessperson-86903
06/09/2022, 12:55 PMI’m curious, what’s the reason for that?We use pulumi to manage our infrastructure, with a custom pants plugin to run the program inside an isolated context orchestrated by pants (pulumi is essentially terraform but you can use python instead of a markup language). Pulumi has a docker.Image which when used will build the docker image and push it to its registry at build time. All we have to do is add the dockerfile as a loose
files()
context as a dependency to the custom pulumi target.
We’ve been doing this before pants rolled out a docker_image
target, and just haven’t felt a compelling reason to switch 🤷agreeable-oyster-28981
06/09/2022, 1:01 PMActually, having to fetch the toolchains every time you run a CI job sounds like a disadvantage. You really might want to rethink how you use your CI, or PantsI think that we’re not looking for the same benefits from pants. the main benefit for us is to have a convenient way to setup a python mono-repo. I gave poetry a try but it was falling behind when trying to integrate with Google Cloud Function and building a docker image. So far, pants have been pretty good for all that. I’ll check what needs to be cached for pants to run properly, but I don’t see any reason why that would not work, every other build tool I’ve used (mainly yarn and npm) works properly with Gitlab caching. Maybe I should have been more accurate about caching. It’s not “artifact caching” per se. Artifact are stored when a job is complated. For the “caching” part, it’s not just the “artifacts” that are cached, but pretty much any files. So
~/.cache/pants
can be added properly
regarding the toolchain, nothing prevent me to have it part of my CI image, I was actually considering testing that at some points.square-forest-55300
06/09/2022, 1:03 PMgitlab/gitlab-runner:latest
agreeable-oyster-28981
06/10/2022, 11:28 AMgitlab/gitlab-runner:latest
didn’t really help.
I just installed docker directly and will mount the socket as suggested by @faint-businessperson-86903
I was trying to follow the docker:dind pattern, using a docker-entrypoint.sh
file, which works if I’m tyring to do direct call to docker, as docker build . xxx
but doesn’t work when using pants through docker, as it bypass the entry point.