<@UB2J9BQA0>: nice work on <https://gist.github.co...
# development
w
@hundreds-father-404: nice work on https://gist.github.com/Eric-Arellano/defca7d864a9f3939448964beea618d4 . i expect that
sys.getsizeof
is lying to you a bit, because it isn't recursive (see https://docs.python.org/3/library/sys.html#sys.getsizeof), but that measuring recursively is challenging here (because things like TransitiveHydratedTarget intentionally structure share, and would lie to a naive recursive size calculation).
objgraph
might help there.
having said that, it's pretty clear that eliminating all cases where we hold FileContent in memory seems like the big one.
stdout/stderr are likely to be problematic as well... we discussed making those lazy on
ExecuteProcessResult
initially, but went for eager in the end.
some of this might boil down to the rust/python boundary, in that when an
intrinsic/builtin
rule "lifts" a "large" type to be fed to a python
@rule
, it should avoid actually persisting that in the
Graph
.
aaaaaand going to bed
❤️ 1
thank you for looking at this!
... hm. having said that. all of the things in the
_handles
map are
Values
held inside the NodeKey/NodeResult enums. so that's not really very helpful.
but still going to bed.
😴 1