hi everyone! really struggling with pex files, and...
# general
l
hi everyone! really struggling with pex files, and not sure I understand the issue here when trying to get a python pex inside a docker image running in an M1 mac. At first I did not use
complete_platform
after errors I tried it. i have a simple server in my BUILD
Copy code
poetry_requirements(
    name="admin-server-requirements",
    source="pyproject.toml",
)

python_sources(
    name="admin-server",
    sources=["src/**/*.py"],
    dependencies=[
":admin-server-requirements"
    ],
)

file(
    name="linux_aarch64_py311",
    source="linux_aarch.json",
)

pex_binary(
    name = "admin-server-package",
    entry_point="src/server.py",
    output_path="dist/admin-server.pex",
        dependencies=[
     ":admin-server",
     ":admin-server-requirements",  # Add this line to ensure all requirements are included
    ],
   complete_platforms=[":linux_aarch64_py311"],
    # layout="packed",
    include_requirements=True,
    include_tools = True,

)


docker_image(
    name="admin-server-docker-image",
    source="Dockerfile",
    dependencies=[":admin-server-package", ":admin-server-requirements"],
    repository="myregistry/admin-server",
    tags=["latest"],
)
but when I run this, I get these errors:
Copy code
(feature/hello-world) ✗ docker run myregistry/admin-server
Failed to find compatible interpreter on path /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/local/bin/python3.11 CPython==3.11.12

No interpreter compatible with the requested constraints was found:

  Failed to resolve requirements from PEX environment @ /root/Library/Caches/pex/unzipped_pexes/1/0ed5aea3efed1aee1ccaf102aa8f672b3d2a227e.
  Needed cp311-cp311-manylinux_2_36_aarch64 compatible dependencies for:
   1: pydantic-core==2.33.1
      Required by:
        pydantic 2.11.3
      But this pex had no ProjectName(raw='pydantic-core', validated=False, normalized='pydantic-core') distributions.
   2: cryptography!=3.4.0,>=3.1
      Required by:
        jwt 1.3.1
      But this pex had no ProjectName(raw='cryptography', validated=False, normalized='cryptography') distributions.
   3: pymongo<5.0,>=3.4
      Required by:
        mongoengine 0.29.1
      But this pex had no ProjectName(raw='pymongo', validated=False, normalized='pymongo') distributions.
   4: wrapt<2,>=1.10
      Required by:
        Deprecated 1.2.18
      But this pex had no ProjectName(raw='wrapt', validated=False, normalized='wrapt') distributions.
   5: wrapt<2.0.0,>=1.0.0
      Required by:
        opentelemetry-instrumentation 0.51b0
      But this pex had no ProjectName(raw='wrapt', validated=False, normalized='wrapt') distributions.
   6: protobuf!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.20.2
      Required by:
        googleapis-common-protos 1.70.0
      But this pex had no ProjectName(raw='protobuf', validated=False, normalized='protobuf') distributions.
   7: grpcio<2.0.0,>=1.63.2
      Required by:
        opentelemetry-exporter-otlp-proto-grpc 1.30.0
      But this pex had no ProjectName(raw='grpcio', validated=False, normalized='grpcio') distributions.
   8: protobuf<6.0,>=5.0
      Required by:
        opentelemetry-proto 1.30.0
      But this pex had no ProjectName(raw='protobuf', validated=False, normalized='protobuf') distributions.
   9: charset-normalizer<4,>=2
      Required by:
        requests 2.32.3
      But this pex had no ProjectName(raw='charset-normalizer', validated=False, normalized='charset-normalizer') distributions.
   10: pymongo<5.0.0,>=4.11
      But this pex had no ProjectName(raw='pymongo', validated=False, normalized='pymongo') distributions.
   11: cryptography>=3.4
      Required by:
        jwcrypto 1.5.6
      But this pex had no ProjectName(raw='cryptography', validated=False, normalized='cryptography') distributions.
the dockerfile is:
Copy code
FROM python:3.11-slim
ENTRYPOINT ["/usr/local/bin/python3.11", "/bin/app"]
COPY dist/admin-server.pex /bin/app
and I am on a Mac M1, and I will need to be building images for x86 and arm, and ive tried a few ways
g
Pex files are just zips, so I'd consider
zipinfo
on the file (or another tool) to check if it includes anything at all.
l
@gorgeous-winter-99296 it definitely includes the files, but this seems to be more about the build platform wheels or something like this
g
So as far as I can tell from the errors you posted, none of those dependencies actually end up in the pex, otherwise it would (iirc) list the invalid variants that weren't picked up.
l
right i see I see. I'll keep on digging, makes sense
Hi @gorgeous-winter-99296 I've re read the docs and tried to use pex again. I've more or less got both complete platforms building, just one small error that I cannot figure out:
Copy code
➜  sentinal git:(feature/hello-world) ✗ pants package ::
23:27:26.66 [INFO] Completed: Building dist/admin-server.pex with 9 requirements: fastapi<0.116.0,>=0.115.6, mongoengine<0.30.0,>=0.29.1, prometheus-fastapi-instrumentator<8.0.0,>=7.0.2, pydantic-settings<3.0.0,>=2.3.4, pydantic<3... (108 characters truncated)
23:27:26.66 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Building dist/admin-server.pex with 9 requirements: fastapi<0.116.0,>=0.115.6, mongoengine<0.30.0,>=0.29.1, prometheus-fastapi-instrumentator<8.0.0,>=7.0.2, pydantic-settings<3.0.0,>=2.3.4, pydantic<3.0.0,>=2.7.4, pymongo<5.0.0,>=4.11, rich<15.0.0,>=14.0.0, structlog<26.0.0,>=25.1.0, uvicorn<0.24.0,>=0.23.2' failed with exit code 1.
stdout:

stderr:
No pre-built wheel was available for dnspython 2.7.0.
Successfully built the wheel dnspython-2.7.0-py3-none-any.whl from the sdist dnspython-2.7.0.tar.gz but it is not compatible with the requested foreign target complete platform cp311-cp311-manylinux_2_28_x86_64.
You'll need to build a wheel from dnspython-2.7.0.tar.gz on the foreign target platform and make it available to Pex via a `--find-links` repo or a custom `--index`.



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
Looks like there is some transitive dependency
dnspython
which only appears when I add
complete_platforms=[ ":linux"]
. my expectations are that this should work. Have you seen this sort of incompatibility before? I attach the platform json that I am using
for context, I am using an M1 mac, but docker images need to be built for x86 linux machines