https://pantsbuild.org/ logo
#general
Title
# general
f

few-arm-93065

02/04/2022, 1:18 AM
Hey folks, I could use some help trying to track down an error with pants trying to install a python package. The package is https://pypi.org/project/pyranges/, which only distributes as source code. My pex_binary target declares a dependency on it, but when I try to run it, this is what I'm getting:
Copy code
ProcessExecutionFailure: Process 'Building src.[...]/exe.pex with 10 requirements: boto3==1.20.21, cryptography==3.4.8, numpy==1.21.4, overrides==6.1.0, pandas==1.3.5, pyranges==0.0.111, scipy==1.7.3, setuptools==60.7.1, toml==0.10.2, types-toml==0.10.1' failed with exit code 1.
stdout:

stderr:
ERROR: Could not find a version that satisfies the requirement pyranges==0.0.111
ERROR: No matching distribution found for pyranges==0.0.111
ERROR: Could not find a version that satisfies the requirement pyranges==0.0.111
ERROR: No matching distribution found for pyranges==0.0.111
pid 34310 -> /Users/jwarwick/.cache/pants/named_caches/pex_root/venvs/7f518a293766277276f42aa896ce105b056e5634/c31a5b9c4ba316e0b508d32fa17316ce30a6f329/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/jwarwick/.cache/pants/named_caches/pex_root --log /private/var/folders/3k/1rb9kxyn35z51bc7ny7z1xgw0000gn/T/process-executionC9Rqy4/.tmp/tmpu542abw0/pip.log download --dest /private/var/folders/3k/1rb9kxyn35z51bc7ny7z1xgw0000gn/T/process-executionC9Rqy4/.tmp/tmph7y1egmo/manylinux2014_aarch64-cp-39-cp39 --platform manylinux2014_aarch64 --implementation cp --python-version 39 --abi cp39 --only-binary :all: boto3==1.20.21 cryptography==3.4.8 numpy==1.21.4 overrides==6.1.0 pandas==1.3.5 pyranges==0.0.111 scipy==1.7.3 setuptools==60.7.1 toml==0.10.2 types-toml==0.10.1 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None
My guess is that this has to do with
--only-binary :all:
. When I try to install this package with pip in a fresh virtualenv, there is no error. Can anyone point me in the right direction?
1
h

happy-kitchen-89482

02/04/2022, 1:36 AM
Hi, your guess appears to be correct.
pyranges
has no wheels, only an sdist, so Pex cannot install it with
--only-binary :all:
.
But we install sdists all the time, so I'm not sure what is up here
Which pants version is this?
f

fresh-cat-90827

02/04/2022, 9:02 AM
hey @few-arm-93065, I’ve had this issue myself. Please see https://pantsbuild.slack.com/archives/C046T6T9U/p1636782893243300 for details, a very useful thread. Could you please share your
pex_binary
target declaration? There is a chance you’ve specified
platforms
tag in your
pex_binary
and then you’d need wheels ready for each platform. Version of Pants would also be helpful to know as well as what machine (OS, architecture) you attempt to produce a
.pex
package. Cheers.
e

enough-analyst-54434

02/04/2022, 4:17 PM
@fresh-cat-90827 is correct. When you see these 5 pip flags you know Pex is being run with
--platform
/
pex_binary.platforms
is configured:
--platform manylinux2014_aarch64 --implementation cp --python-version 39 --abi cp39 --only-binary :all:
And that means "resolve for this foreign platform please" which means the wheels must be available pre-built for that platform (None of Pants, Pex or Pip can cross-build).
f

few-arm-93065

02/04/2022, 4:39 PM
well guessed, thank you - yes, I am using platforms, since these pex's will be used inside docker containers. This is pants 2.9.0 on OSX 12.1.
Copy code
pex_binary(
    name="exe",
    entry_point="src/qc_metrics.py",
    dependencies=[':lib'],
    shebang="/usr/bin/env python3",
    platforms=[
        "current",
        "manylinux2014_aarch64-cp-39-cp39",
        "manylinux2014_x86_64-cp-39-cp39",
    ]
)
Confirming - based on my read of the other thread, if I want to continue to build these PEXs on OSX and linux, the solution is to build out a CI process for the pyranges project locally, and publish those wheels to a local artifact store, for pants/pip to resolve, right? The resolve_local_platforms flag is great for linux, but won't have an effect on OSX.
e

enough-analyst-54434

02/04/2022, 5:06 PM
Exactly.
4 Views