Trying to optimize my pex/docker builds. I'm following the multistage build blog post from https://www.pantsbuild.org/blog/2022/08/02/optimizing-python-docker-deploys-using-pants and I'm running into issues on the docker builds now:

RUN PEX_TOOLS=1 python ./app.deps.pex/ venv --scope=deps --compile /bin/app

is failing with

received exit code 1 during execution of `['/usr/local/python/bin/python3.12', '-s', '-E', '-m', 'venv', '--without-pip', '/bin/app']` while trying to execute `['/usr/local/python/bin/python3.12', '-s', '-E', '-m', 'venv', '--without-pip', '/bin/app']`

and I can't figure out how to debug it further. (Not sure if its a pex issue or a pants issue) I've tried leaving running this command myself directly in the container, and it just gives the same error with no more information. Any suggestions for debugging?

https://github.com/sureshjoshi/perfanity/blob/cached-pex/multistage/BUILD Take a look here, this should work

def split_pex_binary(entry_point: str, **kwargs):
    name = entry_point
    if entry_point.endswith('_entry.py'):
        name = entry_point[:-9]
    pex_binary(
        name=f"{name}.deps",
        entry_point=entry_point,
        execution_mode="venv",
        layout="packed",
        include_sources=False,
        include_tools=True,
    )
    pex_binary(
        name=f"{name}.srcs",
        entry_point=entry_point,
        execution_mode="venv",
        layout="packed",
        include_requirements=False,
        include_tools=True,
    )

generating pex binaries like this, and sure enough, this is generating

blahblahblah.pex

directories (not files)

Ah, interesting, I wonder if the execution mode or layout might automatically unpack it to dist

did check that the directories contained

__main__.py

files and seem to be okay. Also tried removing the execution mode and layout, and it did indeed build executable zipfiles again, but it still led to the same error

if you clone my repo, are you able to package that branch?

no, it does not exist, just tried again with a directory that does exist and same issue

whoa. - I got the same error on my machine when I tried outputting to a directory that I didnt have permissions on, but as soon as I put it in

./app

it worked

I was just thinking permissions. I'm using a "special-org-owned" python base image that has some default user permissions and stuff

Ahhh, gotcha - let me know if it works!

yeah, it worked once I got the permissions stuff out of the way.

COPY --chown

?

yeah, I just mean more "is the security team expecting us to do some particular thing?"

Oh, lol - fair enough 😆