https://pantsbuild.org/ logo
#general
Title
# general
m

miniature-lamp-45970

09/02/2020, 4:59 AM
I have a dependency on torch-geometric which requires specific wheels to be installed depending on the hardware platform as given in https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html. I use poetry to generate requirements.txt file and able to generate it from poetry and it directly uses the https url for the wheel file. When this is used in pants using python_requirements() rule, ends up in a ValueError: Invalid requirement in requirements.txt at line 119 due to value 'https://pytorch-geometric.com/whl/torch-1.6.0/torch_cluster-latest%2Bcpu-cp37-cp37m-macosx_10_9_x86_64.whl'. Is there a support for adding a wheel file url as 3rd party dependency or a workaround for now?
w

witty-crayon-22786

09/02/2020, 6:49 PM
this is not supported right now, but it’s probably not too challenging to do: if this ticket looks like it solves that issue, would you mind bumping it? https://github.com/pantsbuild/pants/issues/3063
m

miniature-lamp-45970

09/02/2020, 9:29 PM
yes, it would solve the issue. And have bumped i up!
w

witty-crayon-22786

09/02/2020, 9:30 PM
thank you!
h

hundreds-father-404

09/03/2020, 6:28 PM
Hey @miniature-lamp-45970, I’ve been looking into this all this morning, and discovered PEP 440 “direct references”, which appear to be roughly an alternate typing of VCS style dependencies. Rather than
Copy code
git+<https://github.com/pypa/pipenv.git#egg=pipenv>
# or
git+<https://github.com/pypa/pipenv.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=pipenv>
You use
Copy code
pipenv@ git+<https://github.com/pypa/pipenv.git>
# or
pipenv@ git+<https://github.com/pypa/pipenv.git@7921be1537eac1e97bc40179a57f0349c2aee67d>
This works today. See https://www.python.org/dev/peps/pep-0440/#direct-references for more guidance. -- Is this acceptable? It saves a ton of headache for us, such as needing to depdend on another 3rd party requirement and making it easier for us to add support for Poetry and Pipenv. -- (I’m also looking for higher level tutorials and am going to update the error message for Pants to point users to this alternative)
m

miniature-lamp-45970

09/03/2020, 8:09 PM
@hundreds-father-404 Tried using the direct reference and used the file protocol.
pip @ file:///3rdparty/torch_cluster-latest%2Bcpu-cp37-cp37m-macosx_10_9_x86_64.whl
Added the above entry to requirements.txt and ended up with the following stack trace
pants.base.exceptions.DuplicateNameError: A target already exists at 'BUILD' with name 'pip' and target type 'python_requirement_library'. The 'python_requirement_library' target cannot use the same name.
h

hundreds-father-404

09/03/2020, 8:12 PM
So, the
pip
is the name of the dependency. Instead of
pip @ file
, you’d want
torch_cluster @ file
, or whatever the req is called
m

miniature-lamp-45970

09/03/2020, 8:12 PM
ha silly me!
h

hundreds-father-404

09/03/2020, 8:13 PM
Totally okay, I’ve been staring at it all day - Python packaging is confusing af. But that’s good feedback that we shouldn’t use
pip
in the example for this new error message I’m writing. Maybe we’ll do
ansicolors
, which is our go to for docs hehe
To confirm, is this working how you’d expect?
m

miniature-lamp-45970

09/03/2020, 8:29 PM
@hundreds-father-404 Yes, just confirmed able to run the binaries. Thanks for taking time to read the spec and help me with it!
❤️ 1
h

hundreds-father-404

09/03/2020, 8:31 PM
Yay! I’ll add you to the PR I’m putting up a in a few minutes for better error messages. We’ll proactively check if it looks like you’re using VCS style, and give a detailed explanation of how to use PEP 440
m

miniature-lamp-45970

09/03/2020, 8:36 PM
Awesome!
h

hundreds-father-404

09/03/2020, 8:53 PM
Thanks for nudging us to fix #3063! I’m glad we’ll have this as part of the 2.0 release. https://github.com/pantsbuild/pants/pull/10728
w

witty-crayon-22786

09/03/2020, 8:53 PM
thank you both!
❤️ 2
h

hundreds-father-404

09/03/2020, 9:22 PM
Docs updated as well: https://www.pantsbuild.org/docs/python-third-party-dependencies#basic-setup Lmk if you have any suggestions
👍 1
5 Views