Check out my blog post on how to optimize your Docker builds of Python apps for speed, build time, and incrementality: https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/ Also reach out if you're interesting in working on cool stuff like this professionally. We're hiring a fellow Build Engineer

Great post! Heads up that the tables are almost invisible (maybe my dark mode system?)

Same here. Dark mode makes the tables really hard to read. But the content is awesome! My mind is now spinning about how I’ll be able to use this in the future.

Very helpful!

@happy-kitchen-89482 on the dark mode tables

Thanks @bitter-ability-32190 - Doing the research on exactly this has been on my todo list for weeks 🙂

Dark mode issue should be fixed

Yep! Just one left

Great, fixed now hopefully

Yep works!

@rhythmic-battery-45198 Happy to be of service! Before this I knew almost nothing about docker so it was really fun to crack it open and get my hands dirty. I have ideas on improvements as well, but like everything, comes with even more tradeoffs

Thanks @bitter-ability-32190 great write up, love the future direction too. 🙏 👍

Fun follow up. Ended up creating macros implementing most of Joshua's blog post and the referenced pex docs. I had an ask for supporting some "light" code obfuscation. I suggested pep-3147 sourceless distribution as a way to add at least some pain in accessing source code. It was pretty easy to modify the macro to optionally remove python sources after compiling the first party pex venv.

# deps build stage
    instructions = [
        f"FROM python:{python_version}-slim as deps",
        f"COPY {_pex_binary_deps_location(container_pex_binaries)} /binary-deps.pex",
        f"RUN PEX_TOOLS=1 {python} /binary-deps.pex venv --scope=deps --compile --collisions-ok /bin/app",
    ]

    # srcs build stage
    instructions += [
        f"FROM python:{python_version}-slim as srcs",
        f"COPY {_pex_binary_srcs_location(container_pex_binaries)} /binary-srcs.pex",
        f"RUN PEX_TOOLS=1 {python} /binary-srcs.pex venv --scope=srcs /bin/app",
        f"RUN /bin/app/bin/python -m compileall -b --invalidation-mode unchecked-hash /bin/app/lib"
    ]
    if sourceless_first_party:
        instructions += [
            f"RUN find /bin/app/lib -name \"*.py\" -type f -delete"
        ]

This would also yield slightly smaller images too, right? No "source" files? 🤔 I Love it!

Yes a drop in the pond with the heavy weight ML third party dependencies 🙂

(same)