https://pantsbuild.org/ logo
#development
Title
# development
c

careful-address-89803

03/06/2023, 1:16 AM
Hey folks, do any of our visualisations of the rule graph have types and rules as vertexes and edges as which types are produced/consumed by which rules? I'm trying to understand how the export goal works, and it would be really nice to have an easy way of seeing that process and not have to search for "which rule returns this type"
c

curved-television-6568

03/06/2023, 4:10 AM
not a graph as such, but perhaps easier to use than simply searching/grepping:
Copy code
pants ExportResult --help-advanced

`pants.core.goals.export.ExportResult` api type
-----------------------------------------------

ExportResult(description: 'str', reldir: 'str', *, digest: 'Digest' = Digest('e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 0), post_processing_cmds: 'Iterable[PostProcessingCommand]' = (), resolve: 'str | None' =
None)

activated by       : pants.backend.python
dependencies       : builtins
                     pants.backend.experimental.go
                     pants.backend.experimental.java
                     pants.backend.python.lint.autoflake
                     pants.backend.python.lint.black
                     pants.backend.python.lint.docformatter
                     pants.backend.python.lint.flake8
                     pants.backend.python.lint.isort
                     pants.backend.python.typecheck.mypy
                     pants.core
                     pants.engine.process
dependents         : pants.backend.python
returned by 3 rules: pants.backend.python.goals.export.do_export
                     pants.backend.python.goals.export.export_tool
                     pants.backend.python.goals.export.export_virtualenv_for_targets
consumed by 0 rules:
used in 4 rules    : pants.backend.python.goals.export.export_tool
                     pants.backend.python.goals.export.export_virtualenv_for_resolve
                     pants.backend.python.goals.export.export_virtualenv_for_targets
                     pants.backend.python.goals.export.export_virtualenvs
works for the rules too 😉
Copy code
pants pants.backend.python.goals.export.export_tool --help-advanced

`pants.backend.python.goals.export.export_tool` rule
----------------------------------------------------

Undocumented.

activated by : pants.backend.python
returns      : ExportResult
takes 1 input: ExportPythonTool
awaits 1 get : Get(ExportResult, VenvExportRequest, ..)
of course, all this is available in json-format (from
pants help-all
), so custom scripting could be applied to turn this into a graph…
heh, which is ~2.5Mb of json by now (or 64511 lines) 😛
c

careful-address-89803

03/06/2023, 4:30 AM
oh yeah, that looks like just what I need. I was digging through the rust code and wasn't finding it easy.
c

curved-television-6568

03/06/2023, 4:32 AM
most rules are on the Python side anyways…
oh, or did you mean in search for some graph output..? 🙂
but, these graphs tend to be BIG. so not always easy to look at 😉
you provide that option when you run some pants goal, and the rule graph used during that run is exported to dot files in the provided directory.
c

careful-address-89803

03/06/2023, 4:43 AM
yeah, I was looking at engin-visualize-to. There are the huge graphs ("graph\.\d{3}\.dot"), but there's also "rule_graph.dot" and "rule_graph.export.dot". Those other 2 are close to what I was looking for, but the types aren't surfaced. The
help-all
goal is actually perfect for converting to a dot for this purpose, all the information I'd want is surfaced for each rule and type
👍 1
b

busy-vase-39202

03/06/2023, 3:19 PM
Not at all an answer to your question, but fyi for help visualizing a graph there is also GraphMyRepo.com. (Caveat: it does not currently support private repos.)
w

witty-crayon-22786

03/07/2023, 6:29 PM
Those other 2 are close to what I was looking for, but the types aren’t surfaced.
so, they used to be. but a recentish rule graph change caused us not to walk through union usages to all of the implementations of the union.
rule_graph.dot
will include everything, and so should show it. but it is too large to be useful.
rule_graph.export.dot
will only show things relevant to
export
, but will fail to walk into unions
i think that https://github.com/pantsbuild/pants/blob/d88e25829160d37aeefab4d7e470db52ccb95aac/src/rust/engine/rule_graph/src/lib.rs#L278-L314 needs to be updated to walk through unions. can file a ticket if it’s something you’re interested in looking at
c

careful-address-89803

03/09/2023, 3:06 AM
peak usefulness
b

busy-vase-39202

03/09/2023, 5:25 AM
If that's graphmyrepo, it's zoomable
4 Views