https://pantsbuild.org/ logo
Join Slack
Powered by
Hey there dear guys! :slightly_smiling_face: I am ...
# general
p
Hey there dear guys! 🙂 I am building a docker image, and works like a charm: 
pants package path/to/src/Dockerfile
- the only issue is that the image is built with architecture 
arm64
instead of 
x86_64
I managed to build the docker image with the right platform by adding this to the FROM directive: 
FROM --platform=linux/amd64 python:3.10
The images builds correctly, but I get this error when running it:
Copy code
Failed to find compatible interpreter on path /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/local/bin/python3.10 CPython==3.10.13
2.)       /usr/bin/python3.11 CPython==3.11.2

No interpreter compatible with the requested constraints was found:

  A distribution for matplotlib could not be resolved for /usr/local/bin/python3.10.
  Found 1 distribution for matplotlib that do not apply:
  1.) The wheel tags for matplotlib 3.7.3 are cp310-cp310-manylinux_2_17_aarch64, cp310-cp310-manylinux2014_aarch64 which do not match the supported tags of /usr/local/bin/python3.10:
  cp310-cp310-manylinux_2_36_x86_64
  ... 839 more ...
I tried to add the complete platform of lambda x86_x64 and reference it, in order to have pip dependencies resolved for x86_64:
Copy code
python_sources(name="server", sources=["**/*.py"], dependencies=["//:env_file"])

python_tests(
    name="tests",
    sources=["tests/**/test_*.py"],
    dependencies=[":tests_support_files", "//:env_file"],
)

resources(
    name="tests_support_files",
    sources=["tests/**/*"],
)

pex_binary(name="bin", entry_point="main.py", complete_platforms=["3rdparty/platforms:aws_lambda_python_3_10"])

docker_image(name="docker")
But this now gives this error when running the container: 
/usr/bin/env: 'python3.9': No such file or directory
The PEX with the complete platform for py3.10 seems to be looking for python3.9 🤔 Does anybody have any solution for this? Would really appreciate some guidance 🙏 thanks so much!
✅ 1
l
Can you post your Dockerfile?
p
Copy code
FROM --platform=linux/amd64 python:3.10

ENTRYPOINT ["/bin/main"]

COPY projects.logo_gen.server/bin.pex /bin/main
Thanks so much 🙏
l
What python interpreter do you want to use?
p
3.10 ideally
l
What interpreter constraints are you using?
I think you have to use something like: 
[">=3.10,<3.11"]
p
interpreter_constraints = ["==3.10.*"]
I have this in my pants.toml
should I change it to 
[">=3.10,<3.11"]
?
l
For me it looks like the pex file is looking for python 3.9 which is not installed in the container.
p
Yep that seems to be happening 🙂 the strange thing is that is looking for py3.9 when I always used 3.10
And it works correctly if I don't specifiy the 
complete_platforms
for the pex in the BUILD file
But if I do, it start searching for py3.9
l
Does the container run correctly, if you just use 
python:3.9
in your docker file?
p
i can try 🙂 was trying to avoid it
because i always run the repo with 3.10
I tried with python 3.9, it's super strange because I get a very similar error:
Copy code
16:45:02.57 [INFO] Completed: Building docker image logo-generator:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Failed to find compatible interpreter on path /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/local/bin/python3.9 CPython==3.9.18
2.)      /usr/bin/python3.11 CPython==3.11.2

No interpreter compatible with the requested constraints was found:

  A distribution for matplotlib could not be resolved for /usr/local/bin/python3.9.
  Found 1 distribution for matplotlib that do not apply:
  1.) The wheel tags for matplotlib 3.7.3 are cp310-cp310-manylinux2014_x86_64, cp310-cp310-manylinux_2_17_x86_64 which do not match the supported tags of /usr/local/bin/python3.9:
  cp39-cp39-manylinux_2_36_x86_64
  ... 766 more ...

  A distribution for matplotlib could not be resolved for /usr/bin/python3.11.
  Found 1 distribution for matplotlib that do not apply:
  1.) The wheel tags for matplotlib 3.7.3 are cp310-cp310-manylinux2014_x86_64, cp310-cp310-manylinux_2_17_x86_64 which do not match the supported tags of /usr/bin/python3.11:
  cp311-cp311-manylinux_2_36_x86_64
  ... 912 more ...
l
I just wanted to see if 3.9 would work without issues. Clearly it does not. I guess you have to work on your complete platform file. Pex is looking for cp39-cp39-manylinux_2_36_x86_64/ cp311-cp311-manylinux_2_36_x86_64. None of those are part of your complete platform file.
p
Sure I can check that thanks a lot 🙏 but still, why is it looking for py3.9 when I am using the complete platform file?
l
I am not sure regarding that. Could you try to add 
cp310-cp310-manylinux_2_36_x86_64
to your complete platform file? And then try to run the python 3.10 container.
p
You mean as part of the "compatible_tags" list? 🙂
Still the same error about py3.9
l
Did you change back the container to 3.10?
p
I did
What should that change do? 🙂
l
I was just looking at your first error. There it was stated that 
cp310-cp310-manylinux_2_36_x86_64
is needed but not found. Adding 
cp310-cp310-manylinux_2_36_x86_64
to the compatible_tags list should solve the issue.
p
Oh i see! that makes sense 🙂 the main problem here is that by adding the complete platform file, then the binary is looking for py3.9 instead of 3.10, so that tag is not even used i think
l
You could also add 
cp39-cp39-manylinux_2_36_x86_64
and try 3.9
p
Still doesn't work 😞 I'm very confused at this point
I am not getting why is it looking for py3.9 or 3.11 when the base image is py3.10 and the complete platform file is for py3.10
The main issue is actually not docker; is the 
pex_binary(name="bin", entry_point="main.py", complete_platforms=["3rdparty/platforms:aws_lambda_python_3_10"])
in the BUILD file If i run 
pants package /path/to/file:bin
I get the same binary file that if I run locally and not in the docker container, I get the same py3.9 error
If the pex binary works, it would work also in docker I believe
l
Why are you actually using the aws lambda complete platofrm file?
p
Because it's the only one I could find to build 
x86_64
which is my target architecture - and I thought it should be the same whether is coming from lambda or not
Is there another way to specify 
x86_64
to the pex binary?
l
Yes, but that one is specifically create for the lambda runtime and that is running on 
public.ecr.aws/lambda/python:3.10
Does your container start if you use that as your docker base image?
p
Trying that right now 🙂 thanks a lot! I also that is possible to specify platforms directly in the pex_binary directive: https://www.pantsbuild.org/docs/reference-pex_binary#codeplatformscode
Not super sure what should I input there though
I still get the py3.9 error even with lambda as base image - as I thought, the problem is not docker, is the pex binary itself
l
I am sadly out of ideas. I actually never work with 
pex_binary
targets 😕 . Are you running the container on your mac?
🙏 1
p
Yep, but in a ubuntu dev container
b
Sorry for the trouble. The lambda complete platform may not be the right one, it be better to generate one that matches the 
Python:3.10
container you’re using… but I suspect that won’t help directly. This looks like it might be https://github.com/pantsbuild/pants/issues/19514 where there’s a suggestion of using the 
shebang
field as a workaround.
🙏 1
❤️ 1
p
Thanks so much @broad-processor-92400! 🙂 I tried the shebang workaround actually and that solved the py3.9, and I also exported the complete platform from inside the docker image, and that works! Sharing infos here if anybody will face similar issues: My BUILD is now:
Copy code
python_sources(name="server", sources=["**/*.py"], dependencies=["//:env_file"])

python_tests(
    name="tests",
    sources=["tests/**/test_*.py"],
    dependencies=[":tests_support_files", "//:env_file"],
)

resources(
    name="tests_support_files",
    sources=["tests/**/*"],
)

pex_binary(
    name="bin", 
    entry_point="main.py", 
    complete_platforms=["3rdparty/platforms:docker_310"], 
    shebang="#!/usr/bin/env python3"
)

docker_image(name="logo-generator")
With the complete platform being this. I followed the guide here to get it (explained in 
Python AWS Lambda zip artefacts
). My docker file is:
Copy code
FROM --platform=linux/amd64 python:3.10

ENTRYPOINT ["/bin/main"]

COPY projects.logo_gen.server/bin.pex /bin/main
🎉 2
3 Views
Open in Slack
PreviousNext
Powered by