https://pantsbuild.org/ logo
Join Slack
Powered by
Hi folks :wave:. I want to create a pex_binary and...
# general
l
Hi folks šŸ‘‹. I want to create a pex_binary and include another pex_binary built by Pants inside of it. The idea is that the initial pex launches a docker container which runs the second pex. What I currently have is:
Copy code
pex_binary(
    name = "client",
    entry_point = "client.py",
    dependencies = [":server"],
)

pex_binary(
    name = "server",
    entry_point = "server.py",
)
but it seems that the server pex is not part of the client pex binary. I also tried wrapping the server pex with 
resource
, but that target only supports files not targets. Is there a way to do this currently?
h
Interesting, I donā€™t think that will work today, but it probably should, in the same way that a docker image (which is a ā€œpackageableā€) can consume a pex_binary, (which is another ā€œpackageableā€)
If you run with 
--keep-sandboxes=always
and look inside the sandbox where the 
client
pex is created, you donā€™t see the server pex in there?
Worth checking in case it ends up in a non-obvious location
l
Nope, don't see it! Thanks though for the suggestion šŸ™‚
Seems that 
python_test
has runtime_package_dependencies which is exactly what I want!
But then for 
pex_binary
šŸ˜
If I run 
pants package //:server
and copy the 
server.pex
into the workspace root and use a resource like this:
Copy code
pex_binary(
    name = "client",
    entry_point = "client.py",
    dependencies = [
        ":server-pex",
    ]
)

resource(
    name = "server-pex",
    source = "server.pex",
)
and package the client pex, the server.pex file is included inside the client.pex!
So either 1. expose 
runtime_package_dependencies
OR 2. support packages inside 
source
attribute for resource targets OR 3. support adding "packageable" sources in the 
dependencies
attribute of pex_binary 4. something else I haven't though of?
You might be more familiar with the codebase @happy-kitchen-89482, what would be the easiest to hack in a fork? If it works, I can try and make a PR
Okay if I do it like this it works, but man this is ugly šŸ¤£. Changing the name of the executable to 
.py
instead of 
.pex
allows it to be included using 
experimental_wrap_as_python_sources
Copy code
pex_binary(
    name = "client",
    entry_point = "client.py",
    dependencies = [
        ":wrap",
    ]
)

pex_binary(
    name = "server",
    entry_point = "server.py",
    shebang = "#!/usr/bin/env python3",
)

shell_command(
    name = "hack",
    command = "cp server.pex server.py",
    tools = [
        "cp"
    ],
    output_files = [
        "server.py",
    ],
    execution_dependencies = [
        ":server"
    ]
)

experimental_wrap_as_python_sources(
    name = "wrap",
    inputs = [
        ":hack",
    ],
)
This is definitely not a long term solution, but works for now
Makes me wonder if I can use 
experimental_wrap_as_resources
Will try that next!
Copy code
pex_binary(
    name = "client",
    entry_point = "client.py",
    dependencies = [
        ":wrap2",
    ]
)

pex_binary(
    name = "server",
    entry_point = "server.py",
    shebang = "#!/usr/bin/env python3",
)

experimental_wrap_as_resources(
    name = "wrap2",
    inputs = [
        ":hack2",
    ]
)

shell_command(
    name = "hack2",
    command = "exit 0",
    output_files = [
        "server.pex",
    ],
    execution_dependencies = [
        ":server"
    ]
)
This works as well
Which is better as we don't have to rename the server.pex to server.py šŸ˜…
Wonder why this doesn't work:
Copy code
pex_binary(
    name = "client",
    entry_point = "client.py",
    dependencies = [
        ":wrap2",
    ]
)

pex_binary(
    name = "server",
    entry_point = "server.py",
    shebang = "#!/usr/bin/env python3",
)

experimental_wrap_as_resources(
    name = "wrap2",
    inputs = [
        ":server",
    ]
)
h
Ah yes, I completely forgot about 
experimental_wrap_as_resources
, glad that works!!
l
Created the following macro:
Copy code
def package_resource(name, source, output_files, description=None, tags=None):
    """
    From Pants Build Slack community: <https://pantsbuild.slack.com/archives/C046T6T9U/p1724752064431699>

    Macro which enables exposing a package target as a resource target.
    This is useful for example wanting to include an archive inside a pex_binary,
    but can be used for any packageable target.
    """
    if tags is None:
        tags = []

    shell_name = f"{name}.command"
    resource_name = name

    shell_command(
        name=shell_name,
        command="exit 0",
        workdir="/",
        output_files=output_files,
        execution_dependencies=[source],
    )

    experimental_wrap_as_resources(
        name=resource_name,
        description=description,
        tags=tags,
        inputs=[
            f":{shell_name}",
        ],
    )
Can be used as follows:
Copy code
pex_binary(
    name = "client",
    dependencies = [
        ":server-pex",
    ],
    entry_point = "client.py",
)

pex_binary(
    name = "server",
    entry_point = "server.py",
)

package_resource(
    name = "server-pex",
    output_files = [
        "server.pex",
    ],
    source = ":server",
)
Works as a charm!
h
Excellent!
Would you be able to write all this up in a discussion under the new ā€œTips and Tricksā€ category? It can be very casual, but I would love to preserve this information for future users.
l
Sure no problem, would be happy to! Any particular format you have in mind?
h
Maybe just a paragraph on what youā€™re trying to do, and how the macro above helped you do it
šŸ‘ 1
l
Let me know what you think
h
Looks great! Thanks!
Open in Slack
PreviousNext
Powered by