https://pantsbuild.org/ logo
Join Slack
Powered by
I'm trying to create a macro for building a docker...
# general
b
I'm trying to create a macro for building a docker image from Python code following https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/ (thanks for publishing it!), and I'm not sure of the best way to get PEX files into the docker image. In particular, if the pex file is at 
abc/def/ghi:target-name
then dep inference will know that 
COPY abc.def.ghi/target-name.pex ...
in the docker instructions refers to it... but how can a macro automatically compute the output path? (particularly the 
abc.def.ghi
part) One option would be to set 
output_path
to have it fixed, but that risks collisions when the macro is used multiple times. (Sketch of the macro in the 🧵 )
✅ 1
👍 1
Copy code
def docker_image_from_pex(*, name, pex_entry_point):
    deps = pex_binary(
        name=f"_{name}-deps",
        entry_point=pex_entry_point,
        complete_platforms=COMPLETE_PLATFORMS,
        include_sources=False,
        include_requirements=True,
        include_tools=True,
        layout="packed",
    )

    srcs = pex_binary(
        name=f"_{name}-srcs",
        entry_point=pex_entry_point,
        complete_platforms=COMPLETE_PLATFORMS,
        include_sources=True,
        include_requirements=False,
        include_tools=True,
        layout="packed",
    )

    # QUESTION: what's the generic form of this?
    output_directory = "abc.def.ghi" # when used in abc/def/ghi... but what if it's used in other/directory too!?

    docker_image(
        name=name,
        instructions=[
            # unpack the external dependencies
            f"FROM {BASE_IMAGE} AS deps",
            f"COPY {output_directory}/{deps.name}.pex /deps.pex",
            "RUN PEX_TOOLS=1 PEX_VERBOSE=2 python /deps.pex venv --scope=deps --compile /opt/venv",
            # unpack our code
            f"FROM {BASE_IMAGE} AS srcs",
            f"COPY {output_directory}/{srcs.name}.pex /srcs.pex",
            "RUN PEX_TOOLS=1 PEX_VERBOSE=2 python /srcs.pex venv --scope=srcs --compile /opt/venv",
            # set up the actual app
            f"FROM {BASE_IMAGE} AS app",
            # ...
            "COPY --from=deps /opt/venv /opt/venv",
            "COPY --from=srcs /opt/venv /opt/venv",
            # ...
        ],
    )
It'd be cool to do something like 
f"COPY {deps.get_me_the_resolved_output_path()} /deps.pex
and thus be totally agnostic to where the 
pex_binary
target decides to put its output, but I cannot find if that's possible
b
Our tools aren't ready to document it yet, but the missing link here is 
build_file_dir()
. It's a 
pathlib.PurePath
to the directory containing the 
BUILD
file
Copy code
f"COPY {'.'.join(build_file_dir().parts) + '/' + name + '-deps-PEX.pex'} /bin/app.pex",
is what ours looks like
b
Cool, thanks! Setting 
output_directory = '.'.join(build_file_dir().parts)
makes the macro work well. How deep into unstable territory am I going by using that?
b
one at all, our help-docs just haven't been made aware of 
objects
(what this is)
Totally stable 🙂
b
awesome
r
Wow! I'm going to copy this. I saw how repetitive this was getting. Thank you!
9 Views
Open in Slack
PreviousNext
Powered by