Okay, sigh... need some assistance with platforms and pex I'm afraid. I'm really trying to shift us to using pex-based docker images; the reduction in code and the reduction in built image size is just compelling... ...but we have several developers that use MacOS. Sigh. Which screws things up considerably. I see a number of solutions, from trying to work out platform strings to creating a Dockerized build platform and building docker within docker using pants (ugh), but I'd love some sort of solution because I'm full sure this is a solved problem. If I'm going to start with the seemingly-obvious, what platform string should I use? I tried looking at pex within my container and under "platform" it said

For the current interpreter at                       /usr/local/bin/python3.9 the full platform string is                        manylinux_2_28_x86_64-cp-3.9.9-cp39

... which is what you get from `pex3 interpreter inspect --all --verbose --indent 4:

{
  "path": "/usr/local/bin/python3.9",
  "version": "3.9.9",
  "requirement": "CPython==3.9.9",
  "platform": "manylinux_2_28_x86_64-cp-3.9.9-cp39",
  "venv": false
}

I tried "pex3 interpreter inspect --markers --tags" and got... well, showers of things, which didn't help. Building my pex setting

platforms

to the above string errored out with

PyYAML==5.4.1

(https://pypi.org/project/PyYAML/5.4.1/#files) which has cp39 manylinux1 and manylinux2014 wheels, so I can't even get past something that seems to have compatible wheels much less the problems I'm going to have when I get to source-only distributions (build a wheel in pants somehow?). I know pants is relatively new to the Docker scene, but "build a pex that can run on docker" seems a useful thing, even if we need to spin up a container to do it. But I can't sell this to my team unless they can build locally (the old version was built entirely within Docker so worked just fine). If that's the way forward, is there a "known best way" to build a pants repository within docker that maintains some of the pants goodness like caching etc?

It turns out the tag you should use for Pex's

--platform

or Pant's `pex_binary.platforms`is `linux_x86_64-cp-3.9.9-cp39`; i.e.: Drop the manylinux bit. Both Pex via

--manylinux

and Pants via

[python-setup] resolver_manylinux

default to assuming foreign Linux platforms are

manylinux2014

compliant / compatible. I.E.: this works

$ pex --platform=linux_x86_64-cp-3.9.9-cp39 PyYAML==5.4.1 --python-shebang "/usr/bin/env python3.9" -opyyaml.pex
$ ./pyyaml.pex -c 'import yaml; print(yaml.__file__)'
/home/jsirois/.pex/installed_wheels/345cdbb6c61904da097ad4b586d9632fac4676c8/PyYAML-5.4.1-cp39-cp39-manylinux1_x86_64.whl/yaml/__init__.py

🙂 What led me down the

pex3 interpreter inspect --markers --tags

was an info string when running the pex executable within a container in a desperate effort to figure something out... Right now I'm running down a rabbit hole with remote execution, which I think is EVENTUALLY going to be the solution (some can build a local buildfarm on their system, others can use a cloud-deployed version, and either way we get what we want: a versioned system for build tools that does something more repeatable). But I'll try the complete-platforms thing in a sec. ...sigh. Ran the

pex3 interpreter inspect...

in the docker container image, did complete_platforms and this time it failed with SQLAlchemy-Paginator, which is a source dist. And we have a few more (including our own git repos). Thanks for the help but it feels like this is a bust for us; perhaps the remote execution path will be more fruitful.

So, this has worked for others (Twittter, 4SQ, ...) , but they have all doubled down and invested in a wheel building infrastructure to pre-build wheels for all platforms they target and then serve those wheels. They then hookup that internal wheel repository with either

[python-repos] repos

or

[python-repos] indexes

depending on how pre-built wheels are hosted. It's really the only way you can use

--platforms

/

--complete-platforms

for any real world projects, you must invest in pre-building wheels.

I can see that, although if I have to double down on building more infrastructure, the buildfarm/REAPI cache seems like better ROI across projects. Ah well. The wheels repo isn't a bad idea either, mind you.

Yeah, pick your poison, but the reapi is more general too. It will work for not-Python issues of same style.

O how I wish I could. I've had to do plenty of "but why doesn't this work on my mac?" support questions. But then I'm choosing to run arch on a homebuilt PC as my daily drive, so I'm just asking for trouble anyway. Seriously though, thanks--that was great support and I learned a lot, and still very impressed with pants. We'll get something going even if I have to back out that change from git in the short term.