I'm running

--stats-memory-summary

on some things and I've gotta say, the numbers don't make a whole lot of sense to me, especially in that the value in bytes doesn't add up anywhere near the memory consumption of the process. I assume I don't have the full picture here? Is this getting shallow size, or not accounting for other memory use in Python, or something else?

I’ve observed something similar. (Not helpful, but at least it’s not something particular to your code…)

it’s only accounting for objects “held by the graph”. it should be doing so deeply, but there are almost certainly gaps. one other potential source of held memory that isn’t accounted for there is our key interning: there is a fairly easy fix for that though, i think: https://github.com/pantsbuild/pants/issues/7675#issuecomment-1281474736

It was off by several orders of magnitude in my case. Like it added up to about 13 MB when the actual memory consumption was 3.9 GB.