I m trying to brainstorm to leverage Pants dep inference + e

Question

I m trying to brainstorm to leverage Pants dep inference + explicit dep modelling and I m wondering if it s possible to run some graph algorithm type stuff on this info An example to find the betweenness centrality to find the most central dependencies Is there a target type that I could `Get` to use this as an input to this calculation

bitter-ability-32190 · Answer

This or this might be a good starting point

bitter-ability-32190 · Answer

FWIW I m probably going to author and maybe publish a plugin that ll give you the dep graph for the entire repo so you could do various calculations on it

flat-zoo-31952 · Answer

Yeah I think that s what I d need There doesn t seem to be something in between `TransitiveTargetsRequest` and `DependenciesRequest` which implies you d just need to recurse and build that graph up

bitter-ability-32190 · Answer

Yeah I don t think there s anything in Pants to do this yet but should be relatively trivial to author

happy-kitchen-89482 · Answer

If you look at the implementation of the `peek` goal it pretty much generates a graph data structure

flat-zoo-31952 · Answer

Wait didn t I write that

flat-zoo-31952 · Answer

Have I already implemented this and just forgot

happy-kitchen-89482 · Answer

fast-nail-55400 · Answer

<https github com pantsbuild pants blob f14f728162f78148cbb21efd649e30378fb59b0f src python pants engine internals graph py L436 L504> has the BFS algorithm used for `TransitiveTargetsRequest`

happy-kitchen-89482 · Answer

You did

happy-kitchen-89482 · Answer

I changed it some but you basically did

happy-kitchen-89482 · Answer

haha

flat-zoo-31952 · Answer

it s been a long time and I have a short memory sweat smile

happy-kitchen-89482 · Answer

You could start by taking the JSON output of `peek` running whatever algorithms on it outside of Pants as an experiment and then bringing the useful ones into a plugin

happy-kitchen-89482 · Answer

But yeah requesting `TargetDatas` gives you what you want

happy-kitchen-89482 · Answer

We can move those outside of `peek py` into a more central location if it turns out that various plugins will find it useful

flat-zoo-31952 · Answer

okay i ll take a look

bitter-ability-32190 · Answer

I have a plugin I was hoping to write that outputs the dep as a graph sounds like `peek` already has this I gotta try it

bitter-ability-32190 · Answer

If so

bitter-ability-32190 · Answer

hold on to your butts

bitter-ability-32190 · Answer

gt Uncaught Invalid DAG structure Found cycle in node path src python pants engine fs py gt src python pants engine engine aware py gt src python pants engine fs py thinking face

bitter-ability-32190 · Answer

So that s interesting The import is under `TYPE CHECKING` but Pants doesn t see that any different

bitter-ability-32190 · Answer

Which makes sense but also cry

hundreds-father-404 · Answer

yeah Pants can t see it differently otherwise MyPy wouldn t work

bitter-ability-32190 · Answer

I ll blacklist this dep

bitter-ability-32190 · Answer

in my script

bitter-ability-32190 · Answer

Nevermind there are several sob

hundreds-father-404 · Answer

yeah lots of import cycles

flat-zoo-31952 · Answer

you can always write a bunch of `Protocol`s to appease the type checker while eliminating import cycles wink weary cold sweat

flat-zoo-31952 · Answer

More seriously though being able to deal with cycles is probably a requirement for this kinda tool because repos that need it will likely have cyclical deps

hundreds-father-404 · Answer

Yeah we went very very out of our way in Pants 2 0 to add cycle tolerance It Was A Thing

hundreds-father-404 · Answer

The tool should adapt to you not you have to adapt to the tool

bitter-ability-32190 · Answer

The tool this is using for rendering does cool things with DAGs shame Pants deps have cycles cry

bitter-ability-32190 · Answer

pants html

hundreds-father-404 · Answer

have you seen CoarsenedTargets It coarsens up cycles to treat them like a single target Needed for JVM compilation