#18936 Some dependencies of pytorch missing from lockfile Issue created by gautiervarjo Describe the bug After adding

torch==2.0.0

to my python requirements and re-generating the lockfile, I encounter missing requirements in the PEX environment when running code or tests:

Failed to resolve requirements from PEX environment @ /home/me/.cache/pants/named_caches/pex_root/unzipped_pexes/2896a51e4b74da2fdbe283e0122b36807e7275ff.
Needed cp38-cp38-manylinux_2_31_x86_64 compatible dependencies for: 
 1: nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux" and platform_machine == "x86_64"                                        
    Required by:                                                    
      torch 2.0.0                                                   
    But this pex had no ProjectName(raw='nvidia-cuda-nvrtc-cu11', normalized='nvidia-cuda-nvrtc-cu11') distributions.                   

<... and many more ...>

Pants version 2.14 In the repro repository below I used

2.17.0.dev4

to check if this was fixed, but no luck. OS Linux, Ubuntu 20.04. Additional info • I made a minimal repository for reproducing the issue: https://github.com/gautiervarjo/pants-torch-missing-reqs • The generated lockfile indeed does not contain the various

nvidia-XXX

requirements, so the error above seems to be a surface-level symptom. • The

METADATA

file inside the downloaded wheel in the pants cache does list these requirements, so I'm not sure why they don't make it into the lockfile. Is it because of those

platform_system

and

platform_machine

attributes?

Requires-Dist: nvidia-cuda-nvrtc-cu11 (==11.7.99) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cuda-runtime-cu11 (==11.7.99) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cuda-cupti-cu11 (==11.7.101) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cudnn-cu11 (==8.5.0.96) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cublas-cu11 (==11.10.3.66) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cufft-cu11 (==10.9.0.58) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-curand-cu11 (==10.2.10.91) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cusolver-cu11 (==11.4.0.1) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-cusparse-cu11 (==11.7.4.91) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-nccl-cu11 (==2.14.3) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: nvidia-nvtx-cu11 (==11.7.91) ; platform_system == "Linux" and platform_machine == "x86_64"
    Requires-Dist: triton (==2.0.0) ; platform_system == "Linux" and platform_machine == "x86_64"

• Installing

torch==2.0.0

in a virtual environment with pip does pull all of these

nvidia-XXX

packages, so everything works. • Previously I had been installing pytorch from the project's own python index. This worked because those wheels bundle all the CUDA they need. But now I'm trying to use the regular PyPI pytorch and running into this issue. Apologies if this is a duplicate issue; I've found plenty of mentions of pytorch and of missing requirements, but nothing that seemed to match! pantsbuild/pants