https://pantsbuild.org/ logo
Join Slack
Powered by
I'm seeing large docker image sizes when using the...
# general
r
I'm seeing large docker image sizes when using the recommended multi-stage build: 
deps.pex/srcs.pex
and running PEXTOOLS with 
venv --compile
. Is this expected and due to the venv-ification? Compared to our baseline of 
pip install -r requirements.txt
, image sizes are going from ~9gb to ~12gb
Dockerfile:
Copy code
FROM vllm/vllm-openai:v0.7.3 as deps
COPY xyz/serve-deps.pex /serve-deps.pex
RUN PEX_TOOLS=1 /usr/bin/python3.12 /serve-deps.pex venv --collisions-ok --scope=deps --compile /bin/app

FROM vllm/vllm-openai:v0.7.3 as srcs
COPY xyz/serve-srcs.pex /serve-srcs.pex
RUN PEX_TOOLS=1 /usr/bin/python3.12 /serve-srcs.pex venv --collisions-ok --scope=srcs --compile /bin/app

FROM vllm/vllm-openai:v0.7.3
RUN apt-get update -y && apt-get install -y awscli

COPY --from=deps /bin/app /bin/app
COPY --from=srcs /bin/app /bin/app
COPY xyz/get_ckpt_and_start.sh /bin/app/get_ckpt_and_start.sh

# RUN pip install --no-cache-dir .
EXPOSE 8000

ENTRYPOINT ["/usr/bin/env"]
CMD ["/bin/app/get_ckpt_and_start.sh"]
w
Can you show your build file? Here’s an example I use https://github.com/sureshjoshi/perfanity/blob/cached-pex/multipex/BUILD#L41
Also, as a quick comparison - you can see what the size/perf effect of —compile is, as a benchmark
r
Docker image is:
Copy code
docker_image(
    name="docker",
    dependencies=[":get_ckpt_and_start"],
    registries=["@aws_ecr"],
    repository="xyz",
    image_tags=["{build_args.GIT_COMMIT}"]
)
w
What about the pex binaries?
c
COPY xyz/serve-deps.pex /serve-deps.pex
To reduce size when all you want is a venv at the end you can do something like 
RUN --mount=type=bind,target=/tmp/bound.pex,source=$PEX_FILE,ro
instead of copying
w
Those aren't retained in the final docker image though, are they?
b
I wouldnt use the vllm base image. I have had a lot more luck using one of the nvidia cuda runtime images and then installing the vllm pip package
The vllm base image is like 8GB and brings in a lot of junk. The cuda runtime images are like 2-4GB
r
Thanks @brief-engine-92399 that makes sense, I had that realization too that since we’re bundling deps in a per, we may not benefit from the preloaded libraries in the same way? Going with nvidia cuda base ended up saving us 3-4 gb on the image size
b
If you aren't gonna use them, then you wont benefit
Congrats on the smaller images 🙂
❤️ 1
9 Views
Open in Slack
PreviousNext
Powered by