https://pantsbuild.org/ logo
Join Slack
Powered by
hi all! we're starting out using pants. trying to...
# general
v
hi all! we're starting out using pants. trying to get 
pants package
, running locally on macos, to successfully build a docker image (linux) with a pex binary inside that has all the right platform-specific wheels for all the 3rd party dependencies in the pex binary. we're running into issues, though, when building the docker container through 
pants package
. this is the setup so far along with all the command output and logging: https://gist.github.com/andreaimprovised/1ef0e1a6a215f9b84d5de9dcecb2d615 . it is complaining "There was 1 error downloading required artifacts" and pex is raising a "MetadataGenerationFailed" exception. i haven't been able to figure out what exactly is wrong any help?
ultimate goal is to run 
pants package
inside circleci's docker executor (circleci does a docker-in-docker build thing) and hopefully 
pants publish
to get the image into AWS ECR. atm, if i can't get 
pants package
to work, i'll probably end up just putting all my code in the Dockerfile, installing pants, and running 
pants package
to build the PEX binary inside the target platform environment. then, i can use the pex binary as the entrypoint
(also apologies if i am unresponsive for the next little while, sleep is calling)
e
So, you use a platform - namely 
"linux_arm64-cp-311-cp311m"
. That's both not correct - there is no such thing as 
cp311m
, CPython dropped the pymalloc 
m
abi tag starting in 3.8 - and it's not a tag of any of the wheels supplied by pydantic-core: https://pypi.org/project/pydantic-core/2.10.1/#files As such, you're forced into attempting a build of the sdist. Pydantic is semi-famous for having gone through a big effort to switch to Rust. Presumably you don't have the cargo toolchain installed on your machine, which, IIUC, is required to build this sdist.
So the correct platform is just 
"linux_arm64-cp-311-cp311"
, but you really, really don't want to be using 
platforms
. Use 
complete_platforms
instead: https://www.pantsbuild.org/docs/reference-pex_binary#codecomplete_platformscode
If you read that complete platforms doc when fresh and rested and follow all the instructions, you will get a machine-generated description of the target docker image that is exactly correct. That might get you around the sdist issue.
v
Thank you, I will give this a go tomorrow :)
e
v
I have changed the platform to 
linux_aarch64-cp-311-cp311
(pydantic does not mark wheels as linux_arm64). the the call to 
pants package
succeeds. the inside of the docker container appears to have the correct wheels
e
Ok, great. Beware though, in the long run a complete platform will serve you better. So make a mental note for when things fail.
v
will do -- our local development workflow doesn't yet pin to a full version of python, just 3.11... sooo, i've been hesitant to go the complete platform route
e
Well, you target a docker image with it, right? That image should have 1 and only 1 interpreter.
v
you know, this is true
e
Exactly.
So complete platforms specifically serve that purpose.
1 complete platform per unique image, that's all you need.
v
just curious, any reason the docs don't discourage 
--platform
?
e
Because you haven't made your 1st contribution yet :)
😆 1
v
fair
e
I made a small nod here in the 
--platform
docs: https://pex.readthedocs.io/en/v2.1.147/buildingpex.html#platform but in general I don't overly proscribe to a fault. Now, I'm not sure what docs you read, likely Pants docs. I'm really just the Pex maintainer guy.
h
This is long overdue, so I came out of semi-hibernation to file this: https://github.com/pantsbuild/pants/issues/19978
Will bang on it next week
(This week is dedicated to preparing my pybay talk, which I am procrastinating on)
@victorious-zebra-49010 if you want to contribute, happy to guide! Most of this would not be difficult.
Open in Slack
PreviousNext
Powered by