https://pantsbuild.org/ logo
#general
Title
# general
m

modern-wolf-36228

11/19/2021, 11:13 AM
Copy code
"Error storing Digest { hash: Fingerprint<8210e4baffa32e2b7f531d76924f977d15dda07da621707329d3d2bdacd7e886>, size_bytes: 2167569207 }: Input/output error"
Is there something I can do if the pex file is huge?
e

enough-analyst-54434

11/21/2021, 6:08 PM
There isn't. That said, Pants 2.7.1+ and 2.8.x both store PEX files in a new way. Instead of storing the whole PEX file, its components parts (user code, PEX bootstrap code, 3rdparty requirements) are all stored separately. So for N requirements, 2+N zips are stored. If you're on Pants older than 2.7.1 or 2.8.x you should be able to upgrade and have that particular problem likely fixed. If you already are upgraded that means you have 1 requirement that's 2GB big on its own, and that we currently have no solution for.
w

witty-crayon-22786

11/29/2021, 6:26 PM
and https://github.com/pantsbuild/pants/pull/13714 (thanks @enough-analyst-54434!) should allow for producing larger PEX “files” (because with
layout="packed"
they will emerge as directories instead). but i’m still interested in tracking down what the internal limitation is here: can follow https://github.com/pantsbuild/pants/issues/13401
m

modern-wolf-36228

11/30/2021, 8:50 AM
thanks, I've subscribed
It will be amazing for ML projects to support it
7 Views