https://pantsbuild.org/ logo
Powered by
f

faint-dress-64989

12/08/2022, 6:45 PM
The 
pex_binary
build option has a param 
restartable
that can be set too true so an application can be restarted automatically when input files changes. This seems to be integrated with the pants run goal. I want to get this same functionality when running applications using docker-compose. Generally I would do this with a volume plus some kind of setting (ie reload w/ 
fastapi
). I'm not sure how to propagate pex changes to docker-compose to restart my container. I was wondering if any one has a solution to this more specifically with a container running fastapi.
e

enough-analyst-54434

12/08/2022, 6:47 PM
So, basically, the only thing you want the pex for is the dependencies IIUC, you want the 1st party code to come from your repo (via a volume mount). Is that about right?
If so 
PEX_INHERIT_PATH=prefer PYTHONPATH=. ...docker commands ...
But the docker commands need to volume mount 
.
too.
f

faint-dress-64989

12/08/2022, 6:51 PM
yeah plus the application to restart automatically
e

enough-analyst-54434

12/08/2022, 6:51 PM
That last bit is not a Pants or Pex thing though - right? Its a uvicorn / fastapi thing IIUC.
f

faint-dress-64989

12/08/2022, 6:51 PM
true
e

enough-analyst-54434

12/08/2022, 6:51 PM
You just have to configure them to restart on changed sources.
f

faint-dress-64989

12/08/2022, 6:52 PM
w/o pants I would do something like:
Copy code
version: "3.9"
services:
  service:
    build: .
    command: uvicorn app.main:app --host 0.0.0.0 --port 80 --reload
    ports:
      - "80:80"
    volumes:
      - ./app:/rate-service/app
    restart: on-failure
e

enough-analyst-54434

12/08/2022, 6:52 PM
Right so so the advice above should allow them to see you changing sources.
Yeah, so advice above and read the cli docs for uvicorn if there are still issues. I had to do that recently to support someone and uvicorn has some path-related args for its wathcing that may be needed.
f

faint-dress-64989

12/08/2022, 6:53 PM
For the 
PEX_INHERIT_PATH
are you referring to a volume for the pex file that gets build in 
dist/
?
e

enough-analyst-54434

12/08/2022, 6:55 PM
Well, 
pex --help-variables
is your friend, but all this is doing is: 1. Breaking PEX hermeticity. PEXes will not allow the environment to leak in by default. 
PEX_INHERIT_PATH=prefer
tells PEX to pre-pend any 
PYTHONPATH
env var set to 
sys.path
2. Adding your volume mounted code to 
PYTHONPATH
If I were you I'd start w/o compose and just a simple PEX file and a simple container and see if you can get that configured to hot reload. Then mix in extra compose complications.
f

faint-dress-64989

12/08/2022, 6:57 PM
I have that working already
e

enough-analyst-54434

12/08/2022, 6:57 PM
Then you have compose issues only it sounds like.
👍 1
f

faint-dress-64989

12/08/2022, 6:57 PM
I'm trying to layer on compose for a more complex dev env so the dev env is as simple as 
docker-compose up
e

enough-analyst-54434

12/08/2022, 6:58 PM
Makes sense. That said, your problem is purely compose at this point.
Sounds like you have PEX + container + hot reload local sources working.
f

faint-dress-64989

12/08/2022, 6:59 PM
Well actually restartable doesnt seem to work for 
docker_image
the container rebuilds, but it doesn't kill the old container meaning I can start the new container on the same port
e

enough-analyst-54434

12/08/2022, 6:59 PM
Ok, then I'd step back and remove Pants and compose
Build a PEX
RUn it in a container with volume mounted local sources and get that working 1st.
f

faint-dress-64989

12/08/2022, 7:00 PM
I can run the pex file with hot reload no issues
e

enough-analyst-54434

12/08/2022, 7:00 PM
Or maybe your point is you expected restartable to work in Pants docker integration
That ... I have no clue about
May be missing feature.
I can run the pex file with hot reload no issues
WHere the PEX file is in the container and the sources you mutate are not, but volume mounted in?
f

faint-dress-64989

12/08/2022, 7:03 PM
Restartable is in the docs, but I think it maybe broken since it doesn't seem to kill the original container. I think a solution with docker compose may be do a dummy run on the pex build on a different port and just mount the path to the pex via compose
I have an example repo: https://github.com/onelive-dev/pants-test/tree/main/src with the container build
e

enough-analyst-54434

12/08/2022, 7:04 PM
Ah, ok I though you meant https://www.pantsbuild.org/docs/reference-pex_binary#coderestartablecode SO, maybe re-post the question explicitly making it clear thats the point of focus. I just muddied things by not understanding that it seems.
Maybe your expectation is 
pex_binary.restartable
auto-propagates to 
docker_image.restartable
and that all works? Sounds reasonable, but I'd break that out in a new thread. Sorry for the noise!
f

faint-dress-64989

12/08/2022, 7:07 PM
I guess my question is more so how people propagate changes from 
pex_binary.restartable
to a docker-compose set up.
My findings have been that 
docker_image.restartable
has a bug, but I don't really need to use if I'm starting the container with docker-compse. I just need to propagate the pex changes. For now I'll just do a 
./pants run
in the background and mount the path to the pex as a volume.
Also before I forget I really appreciate the help!
e

enough-analyst-54434

12/08/2022, 7:11 PM
I don't think I've been much help. I didn't even know Pants had compose integration. I thought it was just building images running them via 
docker {build,run}
f

faint-dress-64989

12/08/2022, 7:11 PM
Pants doesn't have a compose integration
I was main just fishing for ideas for working with pants build artifact and compose
e

enough-analyst-54434

12/08/2022, 7:13 PM
Aha OK. Well restartable is Pants only as you've figured out. So this question is fundamentall how do I get PEX + local sources + hot reload to work with compose, I already have that 3-combo working with straight up docker run.
Is that right?
f

faint-dress-64989

12/08/2022, 7:14 PM
exactly
e

enough-analyst-54434

12/08/2022, 7:15 PM
Ok. Maybe others understand / understood that but not me by a mile. That helps narrow things tremendously.
... and I have no clue - not a compose user for a long time now.
f

faint-dress-64989

12/08/2022, 7:16 PM
Do you mind sharing your full command for 
I already have that 3-combo working with straight up docker run.
?
e

enough-analyst-54434

12/08/2022, 7:16 PM
Whoa - you supposedly have this command right?
I can spend some time and come up with that, but only if needed.
f

faint-dress-64989

12/08/2022, 7:17 PM
Lol no worries. I think I have an idea for how to get this working. I can add my finds to this thread
e

enough-analyst-54434

12/08/2022, 7:17 PM
But, please be clear. You do already have the 3-combo working in docker run or you don't?
f

faint-dress-64989

12/08/2022, 7:18 PM
I do not. I think to get this working I'll need to run pants and docker in tandem
e

enough-analyst-54434

12/08/2022, 7:19 PM
AHA! Ok. My suggestion was to get that working 1st
Way too many moving parts
f

faint-dress-64989

12/08/2022, 7:22 PM
I have tried with this restartable flag, but it doesn't work because pants doesn't kill your old container. So any solution with plain docker or docker compose I think will require a command that look like 
pants run src/python/app: pex && docker-compose up
e

enough-analyst-54434

12/08/2022, 7:23 PM
Ok, I'll let others chime in since you're at the Pants level still. I'd simplify and just use a PEX and get Pants out of the way completely, but as you wish.
That's how I find success debugging anyhow. Simplify, solve, then re-introduce complexity step by step until done.
4 Views
Powered by