https://pantsbuild.org/ logo
Join Slack
Powered by
according to `vmmap` , we were `mmap`’ing 1.7TB… a...
# development
w
according to 
vmmap
, we were `mmap`’ing 1.7TB… and… it was right! https://github.com/pantsbuild/pants/pull/11513
👀 1
h
was this...a regression?
w
no, not as far as i know. we had been overallocating for a long time.
❗ 1
h
so we should prob cherry-pick, possibly as far back as 1.30?
w
we don’t have any known crashers in 1.30/2.0.x/2.1.x, so i don’t think so.
the comment in the change is mostly accurate… this shouldn’t matter… there are just some rare cases where it does, like core dump gathering
👍 1
but will pick to 2.2.x probably
h
Thanks. Somewhat related, 2.1's branch is busted for some reason, and I'm not sure why. I tried creating a fresh PR against it last week and it still failed with the same Git issue that it couldn't find the ref head
a
Commented on the PR - I agree that the 1.7TB should be trivial to shrink down to 110GB or so (yay simple bug!), but I'm not sure about the other shrinking...
(Also I'm fairly sure if someone already has a larger-than-now-configured shared, their new Pants version will insta-crash?)
(I generally somewhat regret choosing LMDB, in retrospect, it's a little too inflexible imo...)
w
makes total sense! thanks for the comment
❤️ 1
yea, the need to shard complicated things. need to re-benchmark at some point.
(Also I’m fairly sure if someone already has a larger-than-now-configured shared, their new Pants version will insta-crash?)
the docs indicate that you can successfully open a larger store… it will only fail if you try to further increase the size. https://docs.rs/lmdb/0.8.0/lmdb/struct.EnvironmentBuilder.html#method.set_map_size
👍 1
Open in Slack
PreviousNext
Powered by