Hi @curved-television-6568! I have some questions regarding building and releasing (deplying) PEX targets. Currently how we do this is we copy the source code into our Dockerfile, and our CD system builds the Dockerfile, and deploys the PEX built inside. This is apparently very inefficient as there is no cache and it always rebuilds it from ground up even when there is no change affecting the PEX. @happy-kitchen-89482 described to me how this will be resolved:

the idea is that there would a build-time container, and the pex would be built in that, and then copied into the deployment image, which is what the

docker_image

target describes

Is this build-time container gonna have all the source code needed and be re-used to build whatever we need to deploy?

Hey Horus!

Currently how we do this is we copy the source code into our Dockerfile, and our CD system builds the Dockerfile, and deploys the PEX built inside.

How are you building the PEX currently? E.g. are you running Pants inside the Docker image

yes

Cool. Yes, Pants will build the

pex_binary

for you beforehand, using the normal Pants cache etc. Then it will copy the file into the Docker image for you. Are you already using the

platforms

field with the

pex_binary

target?

The context here is https://github.com/pantsbuild/pants/issues/13682

Hi @ambitious-student-81104! 🙂 In addition to what Eric and Benjy have already said, I think that in the mean time (until we have proper support for leveraging Docker for running arbitrary tasks) you can volume mount relevant cache directories into your build-time container in order to gain some efficiency back.

To clarify, since this is about running in CI, which is linux, then having a

docker_image

target that depends on a

pex_binary

target should work today: When you build the

docker_image

target Pants will first build the pex natively (which is fine, since that is still linux), and then embed it in the Docker image.

you can volume mount relevant cache directories into your build-time container in order to gain some efficiency back.

not sure how this will work. ci box starts afresh for each run, no? are you suggesting external storage?

> you can volume mount relevant cache directories into your build-time container in order to gain some efficiency back.

not sure how this will work. ci box starts afresh for each run, no? are you suggesting external storage?

ah, right. yes in that case there’d need to be some remote syncing of caches during startup https://www.pantsbuild.org/docs/using-pants-in-ci#directories-to-cache