https://pantsbuild.org/ logo
Join Slack
Powered by
Re: Python application in a Docker target. I'm att...
# general
i
Re: Python application in a Docker target. I'm attempting to use a PEX file target (Flask web server) with a Docker target and can't seem to get the right files into the Pants sandbox.
BUILD
Copy code
python_sources()

pex_binary(
    name="app",
    entry_point="app.py",
    restartable=True,
)

docker_image(
    name="docker",
)

python_requirements(
    name="requirements.txt",
)
Dockerfile
Copy code
# syntax=docker/dockerfile:1

FROM python:3.9

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD [ "python3", "app.py" ]
pants log.txt
pants log.txt
☝️ output of 
./pants run receiver:docker
TL;DR
Copy code
#12 [4/4] RUN pip install -r requirements.txt
#12 sha256:6650d01fe23d0788e183e01f323d9f8332c04cff69701539ecaf7f93b4d272b8
#12 0.938 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
#12 ERROR: executor failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 1
------
 > [4/4] RUN pip install -r requirements.txt:
------
executor failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 1
c
I think the local directory isn't included in the dependencies by default, so there aren't any files sent to the docker context. I think you need to specify them targets for the files and then reference those targets in the 
dependencies
section of the 
docker_image
target. It also puts the files into the filepath by default, so you might have to fiddle with that. The docs show the specifics of that [here](https://www.pantsbuild.org/docs/docker#build-docker-image-example), and also an example of including a PEX. I'm new here, but I hope that helps!
i
I did try to add a file like in the docs for requirements.txt, but it turns out if I remove the pip install instruction the build still fails because the app.py file isn't being copied. I forgot to mention this works as expected with docker commands.
c
If you want to include your python files as files to the docker command (rather than include a PEX), you'd need to also specify them as files. something like 
files(name='my_files', sources=('./**/*',))
is a bit of a hammer, but works (although the files are qualified by their path).
I think there's a cool way to unqualify the path using Pants targets, but I haven't really played with that. I just shimmed it by modifying the COPY instruction like 
COPY ./hellodocker/ .
One idea that helps me think about Pants is that Pants isn't really operating on your directory; it's operating on its view of your directory. Pants uses this in fun ways to make life better. But, sometimes you'll need to help Pants understand what files it should include in that view. So the normal docker commands on the command line work fine because they're running in the actual directory; but Pants might see a directory with only the Dockerfile.
i
Apparently you have to explicitly 
COPY receiver/app.pex .
I would want 
docker build...
commands to behave the same as 
./pants package
because I'm using 
docker compose
when running locally. So, I'm now doing this approach:
Copy code
python_sources()

pex_binary(
    name="app",
    entry_point="app.py",
    restartable=True,
)

docker_image(
    name="docker",
    dependencies=[":files"],
)

files(name="files", sources=["./**/*"])

python_requirements(
    name="requirements.txt",
)
I don't really need the pex_binary but I don't know how to tell tailor to not generate it.
Update: I have this working now and learned more about the 
docker_image
API. https://www.pantsbuild.org/docs/reference-docker_image receiver/BUILD
Copy code
python_sources()

pex_binary(
    name="app",
    entry_point="app.py",
    restartable=True,
)

docker_image(
    name="docker",
    dependencies=[":files"],
    context_root="./",
    restartable=True,
)

files(name="files", sources=["./**/*"])

python_requirements(
    name="requirements.txt",
)
receiver/Dockerfile
Copy code
# syntax=docker/dockerfile:1

FROM python:3.9

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD [ "python3", "app.py" ]
docker-compose.yaml
Copy code
version: "3.8"
services:
  postgres:
    image: postgres:14.2-alpine
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    ports:
      - "5432:5432"
    volumes:
      - postgres:/var/lib/postgresql/data
  receiver:
    container_name: receiver
    build:
      context: ./receiver
    image: receiver
    depends_on:
      - postgres
    ports:
      - 8080:8080
    links:
      - postgres
volumes:
  postgres:
    driver: local
The big change here is that the 
context_root
is now set to the location of the BUILD file rather than the workspace root. Additionally, I need to expose port 8080 and the only way to do that is through the command line:
Copy code
./pants run --docker-run-args="-p 8080:8080" receiver/Dockerfile
h
It's past my bedtime, but I'll ping @curved-television-6568 on this, since his day is about to start, and I will also follow up in the morning
c
Sounds like a way forward was discovered, so I'll put a pin 📌 on this until I get better. 😷
Thanks for your supportive help @careful-address-89803 <3
👖 1
h
Oh sorry to bug you, didn't know you're sick. Get well soon!
👍 1
c
Hi @incalculable-yacht-75851 apologies for the long delay. I guess you discovered that “Apparently you have to explicitly 
COPY receiver/app.pex .
” is not true. I’ll just add a few comments for your various issues you came across, in case it’ll bring better clarity to the way Pants works and how Docker have been integrated with it. From your log output, the best piece of information was this:
Copy code
00:12:23.84 [33m[WARN][0m Docker build failed for `docker_image` receiver:docker. The receiver/Dockerfile have `COPY` instructions where the source files may not have been found in the Docker build context.

The following files were not found in the Docker build context:

  * .
Which indicates that your Docker build context was entirely empty. I take two notes with me from this. 1) We should clearly state the fact that the build context sent to Docker was empty here, and 2) We’re not doing glob matching on the 
COPY
paths against the Docker build context, so you could get false positives presented as files not found. @careful-address-89803 was spot on with is observation that:
the local directory isn’t included in the dependencies by default, so there aren’t any files sent to the docker context.
Pants will only include explicitly or implicitly listed dependencies for the 
docker_image
target in the Docker build context (of which are either packageable or 
files
targets). Currently only paths to packaged 
pex
binaries at their default output path and upstream 
docker_image
targets in the 
FROM
dockerfile instruction are inferred implicitly. The final piece is the 
context_root
you discovered. To work transparently with a plain 
docker build
you want the Docker build context root to match that you provide to 
docker
and by default it is set in Pants to the root of your repo, in order to allow you to include any files from any part of your project regardless of where your 
docker_image
target is defined in your source tree. I hope this is all a bit clearer now. Don’t hesitate to reach out with any further questions 😉
🙌 1
10 Views
Open in Slack
PreviousNext
Powered by