https://pantsbuild.org/ logo
Join Slack
Powered by
Can I build PEX from macOS ARM for Linux? I wanted...
# general
h
Can I build PEX from macOS ARM for Linux? I wanted to experiment with environments, but I’m currently facing an error. The error I’m encountering at the moment is the failed build of the wheel for psycopg2. If I remove the ‘environment’ attribute, the PEX binary will build successfully. However, it will encounter failure when running within a Docker environment. Here are the PEX binary and pants.toml files:
Copy code
# BUILD
pex_binary(
    name="bin",
    dependencies=[":src",":gql_schema"],
    script="gunicorn",
    environment="linux_x86_64"
)

docker_image(
    name="core", 
    dependencies=[":bin"],
    repository="core",
    registries=[
        "@helu_private",
    ],
    image_tags=["latest"]
)
Copy code
# pants.toml
[environments-preview.names]
linux_x86_64 = "//:linux_x86_64"

# BUILD
docker_environment(
  name="linux_x86_64",
  platform="linux_x86_64",
  image="python:3.10-slim",
)
Does anyone know what I might be doing wrong?
w
Not using the docker-side of this, but what about something like:
Copy code
pex_binary(
  name="foo-pex",
  entry_point="foo.main",
  dependencies=[
    ":libfoo", 
  ],
  platforms=["linux-x86_64-cp-311-cp311", "macosx-13.3-arm64-cp-311-cp311",]
)
I use this to run my pexes on Mac and Linux, via a 
scie
for the Python interpreter part though. Shouldn't really be a difference with docker You could also build the pex in docker directly, I believe. I've just not yet done it with Pants
b
Using 
platforms
(or 
complete_platforms
for more precision) is a good approach, but environments should also work… what error are you encountering?
e
Probably psycopg2 needs postgres libs installed but the image is python slim.
w
@high-agent-21601 Which psycopg2 are you grabbing? 
psycopg2-binary
?
e
I think given the multitude of published wheels for that, no. He is definitely grabbing 
psycopg2
and not 
psycopg2-binary
. That fully explains the unspecified error given the OP details fwict.
👍 1
h
I’m using psycopg2=“2.9.6”. And image is python:3.10-slim.
I tried running with platforms specified in pex_binary target, but I am not able to package that binary. It fails with error message: stderr: No pre-built wheel was available for psycopg2 2.9.6. Successfully built the wheel psycopg2-2.9.6-cp39-cp39-macosx_10_9_universal2.whl from the sdist psycopg2-2.9.6.tar.gz but it is not compatible with the requested foreign target abbreviated platform cp310-cp310cp36m-linux_x86_64. You’ll need to build a wheel from psycopg2-2.9.6.tar.gz on the foreign target platform and make it available to Pex via a 
--find-links
repo or a custom 
--index
.
e
@high-agent-21601 python:3.10-slim does not have postgresql libraries.
You should be able to observe the same failure without Pants. Just spin up that image with docker run and try to make a venv and pip install that requirement. That should fail.
h
Basically I need image with postgresql libraries ?
e
Yes. I would stop using Pants 1st and just get this working using docker run + an image that works + Pip install. Once you have that working, return to Pants.
So, in your example above there are 2 different docker images. Why?
Why not use the same image?
Shouldn't you be using "core" ?
h
core is built after I obtain pex file. My idea was to build a pex file that can run my application, simply copy it into docker and run it from there.
e
Ok. Well you'll need an non-core image with postgresql libs. Definitely stop using Pants until you sort that out using docker run and pip install directly.
Pants can't do magic there.
h
I get it and it makes sense. Thank you for your support !
e
@high-agent-21601 is there a good reason you're not using psycopg2-binary instead like @wide-midnight-78598 mentioned?
That has pre-built wheels for ~every target platform you might desire: https://pypi.org/project/psycopg2-binary/2.9.6/#files
Ah, not recommended for prod.
h
Yep, thats why I want psycopg2
w
FWIW https://github.com/psycopg/psycopg2/issues/921 https://github.com/psycopg/psycopg2/issues/1283
[2021] The use of the -binary packages in production is discouraged because in the past they proved unreliable in multithread environments. This might have been fixed in more recent versions but I have never managed to reproduce the failure.
As far as I can tell "in the past" references pre-2019ish. In the absence of any other information, I'm sure it's a safer bet to follow their suggestion and to build from source, but I'm a happy 
-binary
user 🤷
3 Views
Open in Slack
PreviousNext
Powered by