https://pantsbuild.org/ logo
Join Slack
Powered by
Running into a bit of an odd error with torch and ...
# general
w
Running into a bit of an odd error with torch and the default resolve. I have a 
3rdparty
using poetry dependencies. Part of it is a pinned torch version. However, when generating the default lockfile I get
Copy code
.....
reading manifest file '/private/var/folders/xn/yvv0m2ln4dg_vyr8rp8_3xzh0000gn/T/pants-sandbox-hKoUHQ/.tmp/tmpx5gwsdis/build/bashplotlib.egg-info/SOURCES.txt'
writing manifest file '/private/var/folders/xn/yvv0m2ln4dg_vyr8rp8_3xzh0000gn/T/pants-sandbox-hKoUHQ/.tmp/tmpx5gwsdis/build/bashplotlib.egg-info/SOURCES.txt'
creating '/private/var/folders/xn/yvv0m2ln4dg_vyr8rp8_3xzh0000gn/T/pants-sandbox-hKoUHQ/.tmp/tmpx5gwsdis/build/bashplotlib-0.6.5.dist-info'
pid 56141 -> /Users/nasron/.cache/pants/named_caches/pex_root/venvs/df773647d6083b93c874a7ce1b736a536cac4192/a7b95f489b195414cc7f18d4effce60ebe6b9862/bin/python -sE /Users/nasron/.cache/pants/named_caches/pex_root/venvs/df773647d6083b93c874a7ce1b736a536cac4192/a7b95f489b195414cc7f18d4effce60ebe6b9862/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/nasron/.cache/pants/named_caches/pex_root/pip/23.1.2/pip_cache --log /private/var/folders/xn/yvv0m2ln4dg_vyr8rp8_3xzh0000gn/T/pants-sandbox-hKoUHQ/.tmp/pex-pip-log.qge5dq92/pip.log download --dest /Users/nasron/.cache/pants/named_caches/pex_root/downloads/fingerprint_artifact.h8tuifvn --no-deps <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl> --index-url <https://pypi.org/simple/> --extra-index-url <https://python.internal.integrateai.net> --find-links <https://download.pytorch.org/whl/cpu/torch_stable.html> --retries 5 --timeout 15 exited with 1 and STDERR:
  ERROR: HTTP error 403 while getting <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>
ERROR: Could not install requirement torch==1.13.1+cpu from <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl> because of HTTP error 403 Client Error: Forbidden for url: <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl> for URL <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>
That url is indeed forbidden
Copy code
$ wget '<https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>'  
--2024-01-29 10:20:25--  <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>
Resolving download.pytorch.org (download.pytorch.org)... 3.161.213.64, 3.161.213.98, 3.161.213.57, ...
Connecting to download.pytorch.org (download.pytorch.org)|3.161.213.64|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2024-01-29 10:20:26 ERROR 403: Forbidden.
However, if you replace the 
+
with urlencoded 
%2B
it works
Copy code
$ wget '<https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp39-cp39-linux_x86_64.whl>'
--2024-01-29 10:19:31--  <https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp39-cp39-linux_x86_64.whl>
Resolving download.pytorch.org (download.pytorch.org)... 3.161.213.64, 3.161.213.98, 3.161.213.36, ...
Connecting to download.pytorch.org (download.pytorch.org)|3.161.213.64|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 199083436 (190M) [binary/octet-stream]
Saving to: 'torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl.1'

torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl.1   7%[======>                                                                                          ]  14.85M  34.2MB/s
any tips? Note pants.toml has
Copy code
find_links = ["<https://download.pytorch.org/whl/cpu/torch_stable.html>"]
to force cpu installation. pants 
2.19.0
may be related to pip upgrade? I am upgrading from pants 
2.16.0
to 
2.19.0
and I see pip changed versions in 
2.17.0
removing find_links fixes this it seems…although im not sure which torch variant is used.
g
Can you reproduce this if you use 
[python-repos].indexes = ["<https://download.pytorch.org/whl/cpu/>"]
instead? That is what I have in all my repos without any issues for the last year.
w
thanks I’ll try that!
That worked!
however, it seems to break mypy which has a dep on torch for typestubs.
Copy code
$ ./pants check ::                                                  
11:38:45.28 [WARN] DEPRECATED: The autoflake plugin has moved to `pants.backend.python.lint.autoflake` (and from the `fmt` goal to the `fix` goal).
11:38:45.31 [WARN] DEPRECATED: The autoflake plugin has moved to `pants.backend.python.lint.autoflake` (and from the `fmt` goal to the `fix` goal).
11:38:45.90 [INFO] Canceled: Building 52 requirements for requirements.pex from the 3rdparty/python/default.lock resolve: anonlink==0.15.2, azure-identity<2.0.0,>=1.12.0, azure-mgmt-containerinstance<11.0.0,>=10.1.0, azure-monito... (1090 characters truncated)
11:38:46.82 [INFO] Completed: Building mypy.pex
11:38:46.83 [ERROR] 1 Exception encountered:

Engine traceback:
  in `check` goal

ProcessExecutionFailure: Process 'Building mypy.pex' failed with exit code 1.
stdout:

stderr:
Failed to resolve compatible artifacts from lock 3rdparty/python/mypy.lock for 1 target:
1. /opt/homebrew/Cellar/python@3.9/3.9.18/Frameworks/Python.framework/Versions/3.9/bin/python3.9:
    Failed to resolve all requirements for cp39-cp39-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.9/3.9.18/Frameworks/Python.framework/Versions/3.9/bin/python3.9 from 3rdparty/python/mypy.lock:

Configured with:
    build: True
    use_wheel: True

Dependency on torch not satisfied, 1 incompatible candidate found:
1.) torch 1.13.1+cpu (via: torch==1.13.1) does not have any compatible artifacts:
    <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
g
Oh! You're on Mac. There's no +cpu for Mac, because all wheels are CPU-only there (unless you use RocM specifically)
w
ah, is there a way to separate out index use by resolve?
g
So it's saying that for +cpu it only has a Linux wheel for amd64 while your interpreter wants a macosx_14 (arm64)
There is not, which is a shame, because I've been meaning to add it for a long time. What we do at work is have one +cpu resolve and one that is !=1.13+cpu, so explicitly forbidding it
w
ok i think i can try that
k that helped. Although our dev environment doesn’t make this easy. I develop on osx M1, but our CI pipeline is x86 linux. So I get things like:
Copy code
stderr:
Failed to resolve requirements from PEX environment @ /home/circleci/.cache/pants/named_caches/pex_root/unzipped_pexes/e5021c3eacbb44af3d891017188a301d1a50060b.
Needed cp39-cp39-manylinux_2_31_x86_64 compatible dependencies for:
 1: nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cuda-runtime-cu11', normalized='nvidia-cuda-runtime-cu11') distributions.
 2: nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cudnn-cu11', normalized='nvidia-cudnn-cu11') distributions.
 3: nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux"
    Required by:
      torch 1.13.1
since the locking locally I assume isn’t picking the gpu versions of torch. Is my only option to try to relock on a linux envronment? How do we typically deal with cross platform things?
g
Yesh~. But the root cause probably isn't what you think it is. Torch wheels are broken. A sane package would declare platform-specific dependencies with platform markers - 
nvidia-cudnn-cu11 ; platform_system == "Linux"
. Torch does that. But only in the wheels destined for Linux anyways. So if you look at the METADATA in the Linux wheel it correctly mentions the following dependencies:
Copy code
Requires-Dist: typing-extensions
Requires-Dist: nvidia-cuda-runtime-cu11 (==11.7.99) ; platform_system == "Linux"
Requires-Dist: nvidia-cudnn-cu11 (==8.5.0.96) ; platform_system == "Linux"
Requires-Dist: nvidia-cublas-cu11 (==11.10.3.66) ; platform_system == "Linux"
Requires-Dist: nvidia-cuda-nvrtc-cu11 (==11.7.99) ; platform_system == "Linux"
Provides-Extra: opt-einsum
Requires-Dist: opt-einsum (>=3.3) ; extra == 'opt-einsum'
But if you look in the Macosx wheel it only lists this:
Copy code
Requires-Dist: typing-extensions
Provides-Extra: opt-einsum
Requires-Dist: opt-einsum (>=3.3) ; extra == 'opt-einsum'
What you can do is to copy those dependencies to your own requirements with the markers. That should work, I believe.
w
thanks I’ll try that
ok still struggling to get this to work. So I get the same error despite trying to force the packages to be installed during mypy
Copy code
stderr:
Failed to resolve requirements from PEX environment @ /home/circleci/.cache/pants/named_caches/pex_root/unzipped_pexes/5b1140f2a4225222d09ec36b20679d1560a75475.
Needed cp39-cp39-manylinux_2_31_x86_64 compatible dependencies for:
 1: nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cuda-runtime-cu11', normalized='nvidia-cuda-runtime-cu11') distributions.
 2: nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cudnn-cu11', normalized='nvidia-cudnn-cu11') distributions.
 3: nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cublas-cu11', normalized='nvidia-cublas-cu11') distributions.
 4: nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cuda-nvrtc-cu11', normalized='nvidia-cuda-nvrtc-cu11') distributions.
E.g. - this is really only needed (so far) for mypy
Copy code
[mypy]
install_from_resolve = "default"
requirements = [
'nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux"',
 'nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux"',
 'nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux"',
 'nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux"',
 'mypy', 'numpy', 'sqlalchemy2-stubs', 'sqlalchemy[mypy]']
but doesn’t look like it’s being picked up
the default resolve has these
Copy code
circleci@ip-10-0-123-55:~/project$ 2>/dev/null ./pants list 3rdparty | grep nv
3rdparty:reqs#nvidia-cublas-cu11
3rdparty:reqs#nvidia-cuda-nvrtc-cu11
3rdparty:reqs#nvidia-cuda-runtime-cu11
3rdparty:reqs#nvidia-cudnn-cu11
g
Hmm, I'll see if I can repro later. If you have a repository that would be very helpful, but I can probably cobble something together otherwise.
w
Might take me a bit if i can reproduce in a separate space, but if you have some ideas that might go faster.
Unsurprisingly, relocking on ubuntu+intel seems to fix this. I’m going to try to see if i can manually merge the items in the lockfile
so - my current approach is to have separate lockfiles and override the default in CI. A bit of a pain to create the linux/x86 lockfile as I’ll need an x86 env to resolve the correct torch packages.
Hmm, not really working either. back to the issue of being unable to download cpu only packages due to the forbidden error
Copy code
stderr:
There were 2 errors downloading required artifacts:
1. torch 1.13.1+cpu from <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp310-cp310-linux_x86_64.whl>
    ERROR: Could not install requirement torch==1.13.1+cpu from <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp310-cp310-linux_x86_64.whl> because of HTTP error 403 Client Error: Forbidden for url: <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp310-cp310-linux_x86_64.whl> for URL <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp310-cp310-linux_x86_64.whl>
2. torchvision 0.14.1+cpu from <https://download.pytorch.org/whl/cpu/torchvision-0.14.1+cpu-cp310-cp310-linux_x86_64.whl>
    ERROR: Could not install requirement torchvision==0.14.1+cpu from <https://download.pytorch.org/whl/cpu/torchvision-0.14.1+cpu-cp310-cp310-linux_x86_64.whl> because of HTTP error 403 Client Error: Forbidden for url: <https://download.pytorch.org/whl/cpu/torchvision-0.14.1+cpu-cp310-cp310-linux_x86_64.whl> for URL <https://download.pytorch.org/whl/cpu/torchvision-0.14.1+cpu-cp310-cp310-linux_x86_64.whl>
when building the pex for a 
mypy
run
Ok - so managed to work my way around this. Basically, having a separate 
pants.ci.toml
that overrides the default lockfile, and sets the torch cpu index url. Then a script to generate both lockfiles, which happens to work well on osx.
Also, this works with 
2.18.2
but the forbidden issue occurs in 
2.19.0
g
We should fix that, but didn't using 
indexes
solve that part?
w
on osx, it seemed to - but on linux x86 it didn’t. Sorry - i’m moving a lot of dials here so it’s not that clear to me either.
Ok, so I have a working setup with 
2.18.2
even with changing from 
find_links
to 
indexes.add
I still get the same issues. E.g. the diff in my toml files is
Copy code
diff --git a/pants.ci.toml b/pants.ci.toml
index a2860ee19..20316f070 100644
--- a/pants.ci.toml
+++ b/pants.ci.toml
@@ -5,7 +5,7 @@ default_resolve = 'default_linux_x86'
 # Note - this forces pex/pants related builds to use the cpu version of torch
 # Mostly should be ok, since iai_fl_client manually installs gpu version
 # bypassing this.
-find_links = ["<https://download.pytorch.org/whl/cpu/torch_stable.html>"]
+indexes.add = ["<https://download.pytorch.org/whl/cpu/>"]
 
 [python.resolves]
 default_linux_x86 = '3rdparty/python/default_linux_x86.lock'
diff --git a/pants.toml b/pants.toml
index 38c65efb7..037388411 100644
--- a/pants.toml
+++ b/pants.toml
@@ -1,5 +1,5 @@
 [GLOBAL]
-pants_version = "2.18.2"
+pants_version = "2.19.0"
 build_patterns = ["PANTSBUILD", "PANTSBUILD.*"]
 build_file_prelude_globs = ["pants-plugins/macros.py"]
 backend_packages = [
and in the linux CI I get (with 
2.19.0
)
Copy code
18:39:47.60 [INFO] Starting: Building 11 requirements for mypy.pex from the 3rdparty/python/default_linux_x86.lock resolve: SQLAlchemy[mypy], azure-identity, azure-mgmt-storage, azure-storage-blob, mypy, numpy, torch, types-proto... (59 characters truncated)
18:39:47.78 [INFO] Starting: Building 56 requirements for requirements.pex from the 3rdparty/python/default_linux_x86.lock resolve: anonlink==0.15.2, azure-identity<2.0.0,>=1.12.0, azure-mgmt-containerinstance<11.0.0,>=10.1.0, az... (1230 characters truncated)
18:40:05.97 [INFO] Completed: Building 11 requirements for mypy.pex from the 3rdparty/python/default_linux_x86.lock resolve: SQLAlchemy[mypy], azure-identity, azure-mgmt-storage, azure-storage-blob, mypy, numpy, torch, types-proto... (59 characters truncated)
18:40:05.98 [ERROR] 1 Exception encountered:

Engine traceback:
  in `check` goal

ProcessExecutionFailure: Process 'Building 11 requirements for mypy.pex from the 3rdparty/python/default_linux_x86.lock resolve: SQLAlchemy[mypy], azure-identity, azure-mgmt-storage, azure-storage-blob, mypy, numpy, torch, types-protobuf, types-python-dateutil, types-setuptools, types-urllib3' failed with exit code 1.
stdout:

stderr:
There was 1 error downloading required artifacts:
1. torch 1.13.1+cpu from <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>
    ERROR: Could not install requirement torch==1.13.1+cpu from <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl> because of HTTP error 403 Client Error: Forbidden for url: <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl> for URL <https://download.pytorch.org/whl/cpu/torch-1.13.1+cpu-cp39-cp39-linux_x86_64.whl>
I’ll try to get something reproducible, but that may take a while.
g
That would be great! I almost thought this was you as well: https://github.com/pantsbuild/pants/issues/20474, but the patch there should work in your case too.
w
Ah yup thats the same issue
I can wait for the next 2.19 patch, not critical at this point
g
👍 I've filed https://github.com/pantsbuild/pex/issues/2348, hopefully we can find a quick solution for it.
👍 1
g
@wonderful-boots-93625 what did you ever end up doing? I'm running into this EXACT same issue with pytorch.
w
Yea so this is related to https://pantsbuild.slack.com/archives/C046T6T9U/p1709001165905529?thread_ts=1708545375.481409&amp;cid=C046T6T9U and I noted my final fix there (it’s pretty dirty, but it works).
@gentle-flower-25372 here is the file, hopefully there’s a better fix coming later on. I usually do
Copy code
./pants generate-lockfiles && ./fixup_torch_lockfile.py
You’ll likely need to fixup the script for your situation
fixup_torch_lockfile.py
❤️ 1
g
@wonderful-boots-93625 I couldn't get this to work. 😢
w
Boo. What's the issue.
g
Well when I use your script it only adds Mac os and wipes out the others
Overall I have no grasp of what's wrong
the worst part is that I can't seem to determine which pytest_runner.pex is failing. In other words, which python_tests are responsible. So I can't even disable tests lol
w
yea you can add logging to the 
list_packages
to see whats the list being used
g
What is that? I can't find that in the docs google search: pants "list_packages"
also, I'm not using +cpu -- just the "regular" 1.31.1. I'm also confused why generating the pex on linux doesn't solve this.
w
in the script i provided, you can add logging to it to see the list its returning. I meant 
list_packages
function in my scriptu
👀 1
yea if you’re not using the cpu version - this script isn’t for you
g
dangggg
w
you should be able to use a more recent pex version to fix resolving links problems though
g
I don't even know if that is my problem. Links all seem to be there. I am using 2.21.0.dev4 which I looked at the 3rd party requirements and it's using a pex version that has the fix.
Copy code
{
  "artifacts": [
    {
      "algorithm": "sha256",
      "hash": "e0df902a7c7dd6c795698532ee5970ce898672625635d885eade9976e5a04949",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp39-none-macosx_11_0_arm64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "fd12043868a34a8da7d490bf6db66991108b00ffbeecb034228bfcbbd4197143",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "d9fe785d375f2e26a5d5eba5de91f89e6a3be5d11efb497e76705fdf93fa3c2e",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp310-cp310-manylinux2014_aarch64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "393a6273c832e047581063fb74335ff50b4c566217019cc6ace318cd79eb0566",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp310-none-macosx_10_9_x86_64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "0122806b111b949d21fa1a5f9764d1fd2fcc4a47cb7f8ff914204fd4fc752ed5",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp310-none-macosx_11_0_arm64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "0d9b8061048cfb78e675b9d2ea8503bfe30db43d583599ae8626b1263a0c1380",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp37-none-macosx_10_9_x86_64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "f402ca80b66e9fbd661ed4287d7553f7f3899d9ab54bf5c67faada1555abde28",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp37-none-macosx_11_0_arm64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "33e67eea526e0bbb9151263e65417a9ef2d8fa53cbe628e87310060c9dcfa312",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp38-none-macosx_10_9_x86_64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "eeeb204d30fd40af6a2d80879b46a7efbe3cf43cdbeb8838dd4f3d126cc90b2b",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp38-none-macosx_11_0_arm64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "50ff5e76d70074f6653d191fe4f6a42fdbe0cf942fbe2a3af0b75eaa414ac038",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp39-cp39-manylinux1_x86_64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "2c3581a3fd81eb1f0f22997cddffea569fea53bafa372b2c0471db373b26aafc",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp39-cp39-manylinux2014_aarch64.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "6930791efa8757cb6974af73d4996b6b50c592882a324b8fb0589c6a9ba2ddaf",
      "url": "<https://pypi-repo-12345678912.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-repo/simple/torch/1.13.1/torch-1.13.1-cp39-none-macosx_10_9_x86_64.whl>"
    }
  ],
  "project_name": "torch",
  "requires_dists": [
    "opt-einsum>=3.3; extra == \"opt-einsum\"",
    "typing-extensions"
  ],
  "requires_python": ">=3.7.0",
  "version": "1.13.1"
}
It seems to have all of the links
this is the error I'm getting
Copy code
Failed to resolve requirements from PEX environment @ /tmp/pants-sandbox-VLPo7Y/requirements.pex.
Needed cp39-cp39-manylinux_2_35_x86_64 compatible dependencies for:
 1: nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cuda-runtime-cu11', validated=False, normalized='nvidia-cuda-runtime-cu11') distributions.
 2: nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cudnn-cu11', validated=False, normalized='nvidia-cudnn-cu11') distributions.
 3: nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cublas-cu11', validated=False, normalized='nvidia-cublas-cu11') distributions.
 4: nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux"
    Required by:
      torch 1.13.1
    But this pex had no ProjectName(raw='nvidia-cuda-nvrtc-cu11', validated=False, normalized='nvidia-cuda-nvrtc-cu11') distributions.
It's weird cause I had this issue before but I could of sworn I solved it by generating the lockfile on a linux host.
no it doens't matter if I generate on linux or mac os I get the exact same result
btw, how did you generate the known_versions?
w
Which known_versions? Of the pex? That was from the slack thread
g
I just wasn't sure how these are being generated, but honestly I'm not sure it matters for my issue but in general it would be nice to know how to generate that.
4 Views
Open in Slack
PreviousNext
Powered by