I have a repo that uses both pandas and a google library. I'd like to build a pex file that will run on both my mac machine and a linux container image. Since pandas has some bindings under the hood, it requires the pre-built wheels, but google has a core dependency that doesn't have any pre-built wheels (https://pypi.org/project/grpc-google-iam-v1/#files). So when I tried to build the pex for multiple platforms, it tells me that it can't find a matching distribution for that grpc package. I was doing this by specifying something like:

pex_binary(
  name="pex_binary"
  platforms=["linux-x86_64....", "macosx..."]
)

Am I missing some sort of config that would let the google dependency build its own wheel, or does it always have to use a wheel when specifying a platform?

From https://github.com/pantsbuild/pants/issues/12205#issuecomment-861516498

When you specify a platform, all distributions necessary must be available as pre-built wheels. It’s up to you to pre-build them and make them available.

Ah, that answers the question, thanks! The route I currently took is just to have a build container for pants that's the same distro as the one that the pex will run in. I might look into building the wheels, but I feel like that might turn into whack-a-mole. Did you end up having to do it for quite a few packages?

The route I currently took is just to have a build container for pants that’s the same distro as the one that the pex will run in.

Sorry I am struggling to understand what this implies, could you please explain? I’d love to understand your situation better.

Did you end up having to do it for quite a few packages?

We have about 50 external dependencies and I’ve built 5 wheels out of them (hosted on a private PyPI). I do agree that it’s not a sustainable situation to build many wheels (in particular those that are not pure Python — with C or Rust components — because you’ll need a toolchain present to build it for each target architecture/version).

I made a docker-compose to handle packaging.

services:
  build:
    image: "python:3.7"
    volumes:
      - .:/code
    command: sh -c "code/pants package ::"

ah I see. If you ever decide to look at the wheel building, look at https://github.com/pypa/manylinux#docker-images since you are already in Docker space. Good luck!