Hi 🙂 I need some help understanding the correct way to use pants for my project the end result i need is for pants to set up about 8-9 dockers to all run at the same time I have all of the dockers and their relative entry-points (pexes made with pants as well) configured as docker image targets so i could run them using pants run but the pants run command doesn't accept more then one target at a time (a little about the env I'm running pants in a docker and i expect the dockers to work in a "docker in docker" (or "docker out of docker" I'm pretty sure it's the same thing tho) style) Currently i am using a bash script to run things (for target; pants package target; pants run target &) This imposes several problems: Firstly - it forces me to run pants concurrently which disables pantsd, and I would really like to be able use it Secondly - doing it like this doesn't garentee the order in which the dockers are ran and so if one docker requires another to run beforehand it will fail (and they often do since building the targets takes time) Thirdly - for some reason some targets just refuse to build/run when using the script but work just fine if i ran them manually so this is just unreliable Also using pants like this just feels wrong I'd really appreciate it if anyone could help with those problems or show me a new way of looking at this issue

I'm still fairly new to pants, so maybe there's something else to look into, but my gut instinct from your description would be to do the following: 1. Use pants (as you are) to build your docker images:

pants package ::

should build everything in one go 2. Use

docker compose

to bring up all your containers at once. The

docker-compose.yml

file is a place that can hold all your inter-container dependency information like startup order dependencies or mapping common environment variables, etc. a. This is totally independent of pants, and would just "coincidentally" reference your container images as whatever names you are tagging them with in the pants

docker_image

target b. you would run something like

docker compose up

to bring up your stack 3. If needed, you could put the

docker compose up <more options, potentially getting very large>

into a shell script, and then coordinate the running of this shell script using a pants

shell_command

target. a. You could use this to assign dependency information on all your container images, etc. Point 3 is the one I am least sure about, but its what I'm thinking my situation is moving towards

If I understand correctly you are suggesting that i build all the packages(pexes) with pants and then have them as the entry points of the dockers?

right now you have something like (super abbreviated):

python_sources()
pex_binary(entrypoint="app.py")
docker_image(instructions=[... "COPY app_py.pex" ...])

right?

it took me a moment but i understood what you ment in the original message and that's a really good idea i thinks this pretty much solves everything perfectly thanks for the help 😁

no problem. I have plans to do something similar but haven't been able to budget the time yet. I'm interested to know if it works out smoothly for you

i'll test it shortly and update

hahaha that's exactly my situation too

Had a similar situation, one thing that worked better than expected was using pytest-docker-compose to start the docker compose file and then run some tests and have that test target depend on some local-only docker files at latest so that I could run pants test and it would package everything up and run the tests against the composed services