I have a dependency on torch-geometric which requires specific wheels to be installed depending on the hardware platform as given in https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html. I use poetry to generate requirements.txt file and able to generate it from poetry and it directly uses the https url for the wheel file. When this is used in pants using python_requirements() rule, ends up in a ValueError: Invalid requirement in requirements.txt at line 119 due to value 'https://pytorch-geometric.com/whl/torch-1.6.0/torch_cluster-latest%2Bcpu-cp37-cp37m-macosx_10_9_x86_64.whl'. Is there a support for adding a wheel file url as 3rd party dependency or a workaround for now?

this is not supported right now, but it’s probably not too challenging to do: if this ticket looks like it solves that issue, would you mind bumping it? https://github.com/pantsbuild/pants/issues/3063

yes, it would solve the issue. And have bumped i up!

thank you!

Hey @miniature-lamp-45970, I’ve been looking into this all this morning, and discovered PEP 440 “direct references”, which appear to be roughly an alternate typing of VCS style dependencies. Rather than

git+<https://github.com/pypa/pipenv.git#egg=pipenv>
# or
git+<https://github.com/pypa/pipenv.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=pipenv>

You use

pipenv@ git+<https://github.com/pypa/pipenv.git>
# or
pipenv@ git+<https://github.com/pypa/pipenv.git@7921be1537eac1e97bc40179a57f0349c2aee67d>

This works today. See https://www.python.org/dev/peps/pep-0440/#direct-references for more guidance. -- Is this acceptable? It saves a ton of headache for us, such as needing to depdend on another 3rd party requirement and making it easier for us to add support for Poetry and Pipenv. -- (I’m also looking for higher level tutorials and am going to update the error message for Pants to point users to this alternative)

@hundreds-father-404 Tried using the direct reference and used the file protocol.

pip @ file:///3rdparty/torch_cluster-latest%2Bcpu-cp37-cp37m-macosx_10_9_x86_64.whl

Added the above entry to requirements.txt and ended up with the following stack trace

pants.base.exceptions.DuplicateNameError: A target already exists at 'BUILD' with name 'pip' and target type 'python_requirement_library'. The 'python_requirement_library' target cannot use the same name.

So, the

pip

is the name of the dependency. Instead of

pip @ file

, you’d want

torch_cluster @ file

, or whatever the req is called

ha silly me!

Totally okay, I’ve been staring at it all day - Python packaging is confusing af. But that’s good feedback that we shouldn’t use

pip

in the example for this new error message I’m writing. Maybe we’ll do

ansicolors

, which is our go to for docs hehe

@hundreds-father-404 Yes, just confirmed able to run the binaries. Thanks for taking time to read the spec and help me with it!

Yay! I’ll add you to the PR I’m putting up a in a few minutes for better error messages. We’ll proactively check if it looks like you’re using VCS style, and give a detailed explanation of how to use PEP 440

Awesome!

Thanks for nudging us to fix #3063! I’m glad we’ll have this as part of the 2.0 release. https://github.com/pantsbuild/pants/pull/10728

thank you both!

Docs updated as well: https://www.pantsbuild.org/docs/python-third-party-dependencies#basic-setup Lmk if you have any suggestions