https://pantsbuild.org/ logo
Join Slack
Powered by
I’m getting an error trying to build a pex that I ...
# general
f
I’m getting an error trying to build a pex that I don’t understand. I’ve created a minimal repo here: https://github.com/bryanwweber/pants-pytorch. I’m trying to build a pex with a dependency from the Pytorch index via 
pants package <target>
. The final error is:
Copy code
Failed to digest inputs: "Error storing Digest { hash: Fingerprint<4626ecca9d97876686ac89adf4252f9
21617741f8fec8dcb16a8410375d240d8>, size_bytes: 2265668508 }: Invalid argument"
The full error message is on the README in the repo. Any help is appreciated! Thanks!
For what it’s worth, I’m attempting to follow the pattern that Tom introduced here: https://pantsbuild.slack.com/archives/C046T6T9U/p1683222308243009?thread_ts=1683187396.031649&amp;cid=C046T6T9U
e
2.26GB is damn big. I'd guess you're hitting an LMDB storage limit.
f
Interesting, I was wondering if that was the case. Yes, the Pytorch wheels are notoriously large
c
f
I think Tom and I must have the same job 😂
c
looks likely 😂
f
Can confirm that using 2.17.0dev5 fixes the problem. Are there any other workarounds until 2.17 is at least rc?
@bitter-ability-32190 Sorry to keep bugging you 🙂 Since you replied to Tom in that thread above and made the PR that fixed the problem☝️ Do you know if there are any workarounds for files that are too big until 2.17 is more stable?
b
To be very fair, I made the PR in question for reasons unrelated 😂
Also, not bugging at all 🙂 Let me think and try and remember anything I got
Do you use the 
packed
layout option? Or the default 
zipapp
?
f
packed
But this is just going to get expanded into a venv in a docker image, so anything is fine with me
b
SO we use PyTorch at $work, and the GPU-flavored ones too. SO something must be different
f
I’m using pytorch 2.0?
b
So I see the same error. Thats good
But I got nothing 😞
f
For now I’m pip-installing the dependencies to get around pants having to build the pex
b
I mean, yeah that's likely "good enough"
f
Thanks for checking
Open in Slack
PreviousNext
Powered by