I've been trying to introduce `torch-geometric` in...
# general
I've been trying to introduce
into my monorepo, and I've come across an interesting problem. Installing this depends on
(docs here), but the issues arises here with M1 macbooks. There are no official arm64 wheels available on the index. This is usually fine - i can install from source, but
on these packages have a hard dependency on
(link), so trying to do this would lead to
module not found: torch
. I followed the docs for undeclared dependencies, https://www.pantsbuild.org/docs/python-third-party-dependencies#requirements-with-undeclared-dependencies, but this still generates an issue with
python setup.py egg_info
, stack trace:
Copy code
WARNING: Discarding git+<https://github.com/rusty1s/pytorch_scatter@62a958f467bf80c3f566083b4949ec21e2be57d0>. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: Discarding git+<https://github.com/rusty1s/pytorch_sparse@6143af2112cd404adca42111daf14397a45f2d7e>. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement torch-scatter (unavailable)
ERROR: No matching distribution found for torch-scatter (unavailable)
Pinning a single page link source is also problematic since it should build for setups with cuda & without cuda on cpu - there are single page source for each of these cases for the wheels https://data.pyg.org/whl/torch-1.11.0+cpu.html and https://data.pyg.org/whl/torch-1.11.0+cu113.html Any tips/ideas on how to get around this issue? (This usually is no problem in conda because of the package metadata, but there's no way to incorporate conda into pants right?)
Hmm I'm wondering if there is something wrong I'm doing with declaring dependencies. I currently have a requirements.txt in
and the buildfile contains:
Copy code
        "torch-geometric": {"dependencies": [":reqs#torch-scatter", ":reqs#torch-sparse"]},
        "torch-scatter": {"dependencies": [":reqs#torch"]},
        "torch-sparse": {"dependencies": [":reqs#torch"]},
and the requirements.txt have:
Copy code

# also tried installing from source w pep404 syntax
#torch-scatter @ git+<https://github.com/rusty1s/pytorch_scatter@62a958f467bf80c3f566083b4949ec21e2be57d0>
#torch-sparse @ git+<https://github.com/rusty1s/pytorch_sparse@6143af2112cd404adca42111daf14397a45f2d7e>
The Pants documentation you found only deals with handling undeclared runtime dependencies, the issue here is an undeclared buildtime dependency. The fundamental issue here is Pants (Pex) builds distributions in isolation. It does not, mutate a venv like Pip does. In other words, if the very building of a package requires some other package (other than setuptools or wheel), that build must say so. If a
build, this means using `setup_requires`(which has known issues - see the note here) or else using PEP-518 builds where build requirements are handled 1st class. Without this, the build will be happening from a clean slate and it will find no torch present in the build environment.
The only way to work around all this is to pre-build the wheel and host it in your own index or find-links repo and then configure Pants to look at that additional index or find-links repo using https://www.pantsbuild.org/docs/reference-python-repos
I had a hunch that it would come to 1. forking the repo or 2. porting over + adding the arm wheel and hosting them myself.. thanks for the response.