https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

07/13/2022, 7:03 PM
I got a new error from building a pex.
Copy code
Failed to digest inputs: "Error storing Digest { hash: Fingerprint<1872bfe7f1b4c4887a4c1381800062a96d980dea39f700b8ac5cebd0ea613ffe>, size_bytes: 104002766 }: MDB_MAP_FULL: Environment mapsize limit reached
Only thing I could find was a really old bug ticket. Is the proper way to resolve this to delete the cache? This was in our CI system so I suspect our cache just finally got big enough to cause issues (though I'm surprised it happened on the worker it did since we haven't had to clear caches for other workers that are running lots of things like unit tests).
e

enough-analyst-54434

07/13/2022, 7:14 PM
Hrm. When PEXes are too big to fit in an LMDB shard, besides allocating more memory, there is using `layout="packed"`: https://www.pantsbuild.org/docs/reference-pex_binary#codelayoutcode Of course, you wont get a single zip out of
./pants package
, you'll get a PEX packed directory; so not quite as simple to copy a directory around as a single zip file. But big is big and there is not much to be done about that!
I'd be interested to know
du -sh packed.pex
if that works for you.
h

high-yak-85899

07/13/2022, 7:18 PM
hmm, seems like one of our latest builds that worked was 99 MB and this one that failed was 104 MB. So maybe that limit is 100? I don't think that's outrageously big so I'm hesitant to add complexity in how we retrieve/use it.
It's not clear to me how to change that limit
e

enough-analyst-54434

07/13/2022, 7:20 PM
Oh no, that's super small. I was expecting many GBs.
h

high-yak-85899

07/13/2022, 7:21 PM
Yeah, if I'm reading the logs correctly, that's the size of our pex right now 🤷
And all of the default limits in the global options are gigabytes
e

enough-analyst-54434

07/13/2022, 7:23 PM
I think https://www.pantsbuild.org/docs/reference-global#section-local-store-files-max-size-bytes would be the one to adjust. How big is
du -sh ~/.cache/pants/lmdb_store
?
That should get entries GC'd; so it shouldn't grow without bound, but it can get quite big.
h

high-yak-85899

07/13/2022, 8:31 PM
It was 226 GB 😬
w

witty-crayon-22786

07/13/2022, 8:41 PM
mm. yea.
~/.cache/pants/lmdb_store
is currently only GC’d by
pantsd
when it is idle long enough. so it doesn’t do well in ephemeral CI. https://github.com/pantsbuild/pants/issues/11167 covers adding a command for this, but we might honestly want to do something like
git
, where we periodically just say “hold on. we’re going to clean things up synchronously in the foreground”
29 Views