Hi, I'm trying to study how the engine internals w...
# development
l
Hi, I'm trying to study how the engine internals work and stumped about what I am (not) seeing in the graphviz output. I am running pants test in example-python with
--engine-visualize-to
and rendered rule_graph.test.dot with graphviz. I see the root query and the goal_rule for test (screenshot below), but I can't seem to find a node anywhere in the graph referring to the
setup_pytest_for_target
rule that runs at some point to do the pytest-specific work (I can log or debug there and it def runs). Should I expect to see a node for it here? I am also puzzled as to what the
Reentry
nodes mean.
ok, it looks like the rule does show up in the runtime graph outputs, like
graph.003.dot
. These are ... challenging to read. Is there a setting that tweaks the output? If not, I'll take a crack at making it a bit more visually digestible.
🙏 1
some progress ... but I may take another tack because a graph of this size is always unwieldy to look at without the ability to interactively collapse/expand detail ...
w
adjustments to the graph visualization always welcome! i don’t think anyone loves the current display mechanism.
l
maybe promising ...
Screenshot 2023-09-11 at 4.52.27 PM.png
w
ah, neat. yea, that’s a much better visualization for the runtime graph. although it can be tricky since rather than a tree, it is actually a graph (a node might have multiple parents)
l
yes, so that is a node with multiple clients, right? This should be able to accommodate that. The same node with appear several times.
similar to how when debugging in the IDE you would browse an object
w
just know that that can lead to an exponential blowup in the size of the generated data
but yea, if it’s lazily generated maybe.
l
yes, that would be it.
it should be possible to generate a single html that contains the data as json inside and a bit of javascript to generate details as one clicks through
👍 1
otherwise, yes, it would be nuts.
c
There was a way to dump as json all the rules and their Gets and which rules got them. Maybe related to the help goal? I'll try to dig around for it sometime this week
🙏 1
l
slightly more progress, now with lazy loading so full graph can be seen. playing with showing the result type - at least easy to read the function body and see which functions are being called to satisfy the Gets
Ok, packaged up this hacked-together tree-like viewer of the runtime graph .dot files. https://github.com/gauthamnair/pants-graph-view Should be straightforward to run if you are interested, instructions included. There's also an example of the output html here which one can download and poke around.
🎉 3
❤️ 1
c
dumping all the rules and such is just
./pants help-all
, I thought it was something more complicated
🙏 1
l
thank you, I will need to use that too. My next step is trying to understand the relationship between the rule graph and the runtime graph.