according to `vmmap` we were `mmap` ing 1 7TB and it was rig

Question

according to `vmmap` we were `mmap` ing 1 7TB and it was right <https github com pantsbuild pants pull 11513>

hundreds-father-404 · Answer

was this a regression

witty-crayon-22786 · Answer

no not as far as i know we had been overallocating for a long time

hundreds-father-404 · Answer

so we should prob cherry pick possibly as far back as 1 30

witty-crayon-22786 · Answer

we don t have any known crashers in 1 30 2 0 x 2 1 x so i don t think so

witty-crayon-22786 · Answer

the comment in the change is mostly accurate this shouldn t matter there are just some rare cases where it does like core dump gathering

witty-crayon-22786 · Answer

but will pick to 2 2 x probably

hundreds-father-404 · Answer

Thanks Somewhat related 2 1 s branch is busted for some reason and I m not sure why I tried creating a fresh PR against it last week and it still failed with the same Git issue that it couldn t find the ref head

average-vr-56795 · Answer

Commented on the PR I agree that the 1 7TB should be trivial to shrink down to 110GB or so yay simple bug but I m not sure about the other shrinking

average-vr-56795 · Answer

Also I m fairly sure if someone already has a larger than now configured shared their new Pants version will insta crash

average-vr-56795 · Answer

I generally somewhat regret choosing LMDB in retrospect it s a little too inflexible imo

witty-crayon-22786 · Answer

makes total sense thanks for the comment

witty-crayon-22786 · Answer

yea the need to shard complicated things need to re benchmark at some point

witty-crayon-22786 · Answer

gt Also I m fairly sure if someone already has a larger than now configured shared their new Pants version will insta crash the docs indicate that you can successfully open a larger store it will only fail if you try to further increase the size <https docs rs lmdb 0 8 0 lmdb struct EnvironmentBuilder html method set map size>