We’ve got a step in our build process (for docker artifacts) that copies files down from S3 copying them into the docker container. We’ve been looking at the

experimental_run_command

target as a solution for this but have been struggling. Does the

experimental_run_command

support a workflow that performs the following steps? 1.

aws s3 sync <s3://bucket/dir/> ignored_path_in_monorepo/

(or a shell script that runs this) 2. Make those files available to the

docker_image

sandbox 3) A plain old

COPY ignored_path_in_monorepo/ /path_in_docker_image/

in the Dockerfile

Pants 2.16 allows you to install arbitrary URL handlers which can act as middleware for requests. The one implementation we ship with this feature is AWS S3 field downloading support. So once enabled, you'd be able to use an S3 URI as the source of an http_source source object in a

file

target

Thank you! I’ll dig around.

Let me know if that works for you!

https://github.com/pantsbuild/pants/blob/a39c1b970be2cb4421b20728c21c3222aeb4b015/src/python/pants/core/target_types.py#L164-L169 https://github.com/pantsbuild/pants/tree/a39c1b970be2cb4421b20728c21c3222aeb4b015/src/python/pants/backend/url_handlers/s3

Oh yeah, whoops. 😅

Cool, looks promising. We're getting along fine for now with some

resource

targets that are synced manually before pants runs. Excited for this new feature!

Pants should be able to handle any out-of-band pre-steps, so if you have more let us know! We can figure out how to make it better 🙂