https://pantsbuild.org/ logo
Join Slack
Powered by
So, i thought i managed to get torch to work with ...
# general
w
So, i thought i managed to get torch to work with some of the pex cli changes, but sometimes on my M1 I get
Copy code
Dependency on torch not satisfied, 1 incompatible candidate found:
1.) torch 1.13.1+cpu does not have any compatible artifacts:
    <https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp39-cp39-linux_x86_64.whl>
    requirers:
    via: torch==1.13.1
    via: opacus==1.1.3 -> torch>=1.8
    via: pytorch-ignite<0.5.0,>=0.4.11 -> torch<3,>=1.3
    via: torchvision>=0.14.1 -> torch
despite lock succeeding. in my pants.toml I do have
Copy code
[python-repos]
indexes.add = [
   "<https://download.pytorch.org/whl/cpu/>"]
I thought I had this sorted out, until this error started occuring again….
Any. guidance on this? I realized I can reproduce using a pex3 export as well, I figured its doing the same thing
Copy code
$ pex3 lock export 3rdparty/python/tmp_default.lock --resolve-local-platforms
Failed to resolve compatible artifacts from lock 3rdparty/python/tmp_default.lock for 1 target:
1. /Users/nasron/.pyenv/versions/3.9.10/bin/python3.9:
    Failed to resolve all requirements for cp39-cp39-macosx_14_0_arm64 interpreter at /Users/nasron/.pyenv/versions/3.9.10/bin/python3.9 from 3rdparty/python/tmp_default.lock:

Configured with:
    build: True
    use_wheel: True

Dependency on torch not satisfied, 1 incompatible candidate found:
1.) torch 1.13.1+cpu does not have any compatible artifacts:
    <https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp39-cp39-linux_x86_64.whl>
    requ
irers:
    via: torch==1.13.1
    via: opacus==1.1.3 -> torch>=1.8
    via: pytorch-ignite<0.5.0,>=0.4.11 -> torch<3,>=1.3
    via: torchvision>=0.14.1 -> torch
c
I don't use torch personally, but have you already seen https://github.com/pex-tool/pex/releases/tag/v2.1.162 ?
w
Yup! I was a victim and contributing reporter for that, so i have
Copy code
[pex-cli]
# Pin this version manually until the version of pex in <https://github.com/pantsbuild/pex/issues/2348>
# is integrated into pants.
known_versions = [
  "v2.1.162|macos_arm64|95babb1aa147e2803b7744ba0c025aede8e5fc00322ed535675a51563672486b|3676166",
  "v2.1.162|macos_x86_64|95babb1aa147e2803b7744ba0c025aede8e5fc00322ed535675a51563672486b|3676166",
  "v2.1.162|linux_x86_64|95babb1aa147e2803b7744ba0c025aede8e5fc00322ed535675a51563672486b|3676166",
  "v2.1.162|linux_arm64|95babb1aa147e2803b7744ba0c025aede8e5fc00322ed535675a51563672486b|3676166"
]
version="v2.1.162"
in my pants.toml file
some progress - looks like for some reason the 
generate-lockfile
is not resolving my correct platform, If i edit the lockfile, I can export it.
Copy code
diff --git a/3rdparty/python/default.lock b/3rdparty/python/default.lock
index 3096c1798..7f45f293e 100644
--- a/3rdparty/python/default.lock
+++ b/3rdparty/python/default.lock
@@ -5802,6 +5802,11 @@
               "algorithm": "sha256",
               "hash": "71636a5c21927236f4974d2355fb3f66a0b707c28219b0135ff65ed0f0e61287",
               "url": "<https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp39-cp39-linux_x86_64.whl>"
+            },
+            {
+              "algorithm": "sha256",
+              "hash": "e0df902a7c7dd6c795698532ee5970ce898672625635d885eade9976e5a04949",
+              "url": "<https://download.pytorch.org/whl/cpu/torch-1.13.1-cp39-none-macosx_11_0_arm64.whl>"
             }
           ],
           "project_name": "torch",
I’m running this on M1, and so its curious why the lockfile generation resolves to an x86 version
I bet it has something to do with the 
+cpu
suffix
m
Yes, this is due to some strangeness with how torch names its packages. We managed to resolve it by just downloading the wheel and rename it to have this +cpu suffix in the version and upload it to our own wheel repo instead.
w
Alrighty I may do that if I don’t find a cleaner fix.
So - in the end I wrote a script to scrape the links from download.pytorch.org/whl/cpu and then replace the artifacts in the lockfile - after 
generate-lockfiles
It’s kinda ugly, but simplest way forward for me for now. My eyes bleed for this but ah well.
3 Views
Open in Slack
PreviousNext
Powered by