Hey there! Here with another (gitlab) CI question. One of our targets is a Docker image, which we publish to an internal registry. We would like to build the image in one stage (with

pants package

) and then publish in a later stage (with

pants publish

). However, I'm not sure what I need to pass around as an artifact for Pants not to rebuild the image when calling

pants publish

in the second stage. (Fwiw the rationale is the the packaging, which build a bunch of other stuff too, happens in parallel to testing, while the publishing requires the tests to be successful.)

Unfortunately pants' interactions with the external docker service are hard to make cacheable (e.g. what if someone deleted the image from the docker service between the

pants package

and

pants publish

calls... presumably your CI won't do this, but pants can't know that, and has to assume the worst). This means, as you observe, every time the image is requested from pants, it has to be rebuilt, rather than served from cache, like other artifacts. https://github.com/pantsbuild/pants/issues/18287 covers improving this (and links to other related issues too). Currently, I think there's a few options: 1. tweak the docker build to run as fast as possible the second time: (of course, there's a limit on how fast this can actually get) a. make sure docker's own cache is able to help the second build be faster (e.g. deterministic inputs. Generally pants should be okay at this, but you can confirm by setting

[docker] build_verbose = true

in

pants.toml

and looking at the output closely) b. do as much work with pants tools that can be fully cached (e.g.

shell_command

and

adhoc_tool

), and then keep the docker image building to "copy files in", as much as reasonable. 2. use a normal

docker push

rather than

pants publish

. The

pants package

call will output json files into

dist/

that might be able to help with this Thoughts?

yeah, I considered #2, but obviously the nice thing of

pants publish

is that it figures out for me which images need publishing 😅

Yeah that’s definitely convenient. I imagine one should be able to emulate it with a script that finds the

.docker-info.json

files in

dist/

but it’s nice when pants does it…

Btw, quick update - in the end I'm giving up on the separation and just running

pants publish --filter-target-type=docker_image

. What I do is I prebuild other distributables in a first step with

pants --filter-target-type=... package

which I can then deploy in a separate one, and just take the "speed hit" of also building the images in the publish step later.