Hey all, I've got an interesting issue. I have an executable that depends on stream-chat. As you can see from the PyPi page it's OS independent. If I

./pants run

the executable it runs fine. However, I'm trying to build a cross-platform

.pex

with:

platforms = [
      'linux-x86_64-cp-37-m',
      'macosx-10.14-x86_64-cp-37-m'
   ]

and I get

stderr:
ERROR: Could not find a version that satisfies the requirement stream-chat
ERROR: No matching distribution found for stream-chat
pid 181794 -> /home/oliver/.pyenv/versions/3.7.10/bin/python3.7 /home/oliver/.cache/pants/named_caches/pex_root/pip.pex/cf4106b4c7898b462f7c172dde686a9747103f1a --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-execution11pC41/.tmp/tmpgo8mme15/pip.log download --dest /tmp/process-execution11pC41/.tmp/tmp3aui1xab/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt firebase-admin google-cloud-firestore pydantic stream-chat typing_extensions --index-url <https://pypi.org/simple/> --find-links <https://us-central1-python.pkg.dev/infrastructure-314723/python-deps/> --retries 5 --timeout 15 exited with 1 and STDERR:
None

why would it have trouble getting

stream-chat

when it's OS independent? Note that if I remove the

macosx-10.14...

line from

platforms

the

./pants package

succeeds.

Hmm, weird. I would imagine this is a common scenario!

Thanks @happy-kitchen-89482

Which version of Pants is this?

2.4.0

What's the other bug?

And you're running this on linux, it sounds like?

I think this is the same experience I had here? https://pantsbuild.slack.com/archives/C046T6T9U/p1628260681111400?thread_ts=1628260681.111400&cid=C046T6T9U Should we expect pex to be able to build cross platform for non-wheel dependencies? I was flipped in this case. On Mac, building for Linux.

It is similar. But in that case the google grpc library contained native code (I think) whereas this sdist is pure python (I think). But this is where my ignorance comes in - maybe it is not possible to use an sdist in a cross-platform way even if it is pure python. @enough-analyst-54434 would know. I will look a little deeper.

i think that Chris is right: you need a wheel for the relevant platform, rather than an sdist

Chris and Stu are right. The relevant pip flag from the original command is

--only-binary :all:

. That's required when passing the 4 platform bits.

@happy-kitchen-89482 @witty-crayon-22786 thanks for your help! Do you need me to re-run with the

-ldebug

or have you determined that the issue is the

--only-binary

flag? If it's that flag what's the workaround for libs like this that don't provide a wheel? And if it is that flag why was @happy-kitchen-89482 able to run the command without the

constraint.txt

but with all the other flags and have it succeed?

Benjy's results don't make sense. Pex has a mode enabled with

--resolve-local-platforms

that Pants doesn't use IIRC that can turn one of your platform arguments into a normal local resolve, but the other will still be foreign and so wheels cannot be built. To work around, you need to pre-build the needed wheels by hand for each required platform and serve them up from your own cheese shop or find links repo. You then add that repo via

[python-repos]

indexes

or

repos

list options: https://www.pantsbuild.org/docs/reference-python-repos

Ah, I was not passing

--only-binary :all:

This is a limitation of Pip. If that Pex command works for you, that seems like a bug.

yeah, that's very old. Per above my

constraints.txt

has

stream-chat==3.11.0

.

Aha, sorry, missed that bit. Yeah, so that's the issue - that pin. Which leads to the current Pip limitation inherited by Pex for foreign platform resolves.

Bummer! I wonder if it'd be possible (or advisable) to put a workaround in Pants; have it download sdists, build wheels, and create a local cheese shop before calling out to pex. Sounds like more than a small amount of work however. Thanks for all your help @enough-analyst-54434

Yeah, sounds like more of a feature of pip than a bug in Pants, alas.