Total noob question but I just can t wrap my head around thi Pants #general

Total noob question, but I just can't wrap my head...

fast-refrigerator-12613

07/25/2023, 11:22 AM

Total noob question, but I just can't wrap my head around this one yet. Background I'm looking to introduce pants into our python ecosystem, but I haven't been able to figure how to reliable build artifacts for both M1 macs and our linux-based docker containers (aarch64). We run test versions of our system locally for testing and what-not, so it's important to execute the docker containers locally (on m1). The problem seems to stem from how I am specifying the

pex_binary

target. I originally started with

Copy code

pex_binary(
    name="bin",
    entry_point="foo.main",
    shebang="#!/usr/bin/env python3",
    platforms=[
        "macosx-arm64-cp-310-m",
        "linux-x86_64-cp-310-m",
    ],
)

This worked on

2.15.0

in some of my simple POC repos, but then broke in

2.16.0

. In particular, I was unable to install

aiohttp==3.8.5

and

httptools==0.6.0

. Then, I updated the

platforms

to the following:

Copy code

pex_binary(
    name="bin",
    entry_point="foo.main",
    shebang="#!/usr/bin/env python3",
    platforms=[
        "macosx_11_0_arm64-cp-310-cp310",
        "linux_aarch64-cp-310-cp310",
    ],
)

This allowed me to install the two troublesome packages, but then I was unable to build (or find a binary) for

grpcio

. I still haven't found a way around that one yet. Question How are people dealing with building for multiple platforms on an M1? Are there good resources to which you can point me? Am I writing the platforms strings incorrectly? Should I be using

complete_platforms

instead? I'm really at a loss here ``````

fast-refrigerator-12613

07/25/2023, 12:04 PM

Okay - classic example of asking a question then (almost immediately) figuring it out. First, I had missed this message from @enough-analyst-54434 (Thanks again, John!), which suggests that

complete_platforms

are definitely the way to go. Then, I didn't really understand the PEX docs, even though the answer was right in front of me. Anyway, here is a quick summary for anyone else that comes across this so that they don't have to needlessly spin their wheels.

Copy code

# install pex if you don't have it
$ pip install pex

# have pex create the spec for you
$ pex3 interpreter inspect --markers --tags | jq > platforms/platform_mac.json

Then, in generate one for your docker container (or other system)

Copy code

$ docker run --rm -it --entrypoint /bin/bash foo-bar
foo@bar:/# pip install pex
foo@bar:/# pex3 interpreter inspect --markers --tags

I just copy and pasted the output to a file. You can probably find a more fancy way if you really want. Create a

BUILD

file so that Pants knows about those files

Copy code

# platforms/BUILD
file(name="complete_platform_mac", source="platform_mac.json")
file(name="complete_platform_docker", source="platform_docker.json")

Add those references to your

pex_binary

target

Copy code

pex_binary(
    name="bin",
    entry_point="foo.bar.main",
    shebang="#!/usr/bin/env python3",
    complete_platforms=[
        "platforms:complete_platform_mac",
        "platforms:complete_platform_docker",
    ]
)

refined-addition-53644

07/25/2023, 12:22 PM

This is also documented here. I know not the most appropriate location but these kind of issues keep popping up and people are unsure of where to add it in the docs. Thanks to @broad-processor-92400 who has added some of these "recipes" https://github.com/pantsbuild/pants/discussions/18756

👍 2

Open in Slack

Previous Next