https://pantsbuild.org/ logo
#general
Title
# general
f

flat-zoo-31952

02/24/2022, 3:52 PM
I'm trying to brainstorm to leverage Pants' dep inference + explicit dep modelling, and I'm wondering if it's possible to run some graph algorithm type stuff on this info? An example: to find the betweenness-centrality to find the most central dependencies. Is there a target type that I could
Get
to use this as an input to this calculation?
FWIW I'm probably going to author (and maybe publish) a plugin that'll give you the dep graph for the entire repo so you could do various calculations on it
👍ðŸŧ 1
f

flat-zoo-31952

02/24/2022, 4:00 PM
Yeah I think that's what I'd need. There doesn't seem to be something in between
TransitiveTargetsRequest
and
DependenciesRequest
which implies you'd just need to recurse and build that graph up
b

bitter-ability-32190

02/24/2022, 4:01 PM
Yeah. I don't think there's anything in-Pants to do this yet, but should be relatively trivial to author
h

happy-kitchen-89482

02/24/2022, 4:01 PM
If you look at the implementation of the
peek
goal, it pretty much generates a graph data structure
👀 1
f

flat-zoo-31952

02/24/2022, 4:02 PM
Wait didn't I write that?
ðŸĪĢ 1
Have I already implemented this and just forgot?
f

fast-nail-55400

02/24/2022, 4:02 PM
h

happy-kitchen-89482

02/24/2022, 4:02 PM
You did!
I changed it some, but you basically did
haha
f

flat-zoo-31952

02/24/2022, 4:03 PM
it's been a long time and I have a short memory 😅
ðŸĪŠ 1
h

happy-kitchen-89482

02/24/2022, 4:03 PM
You could start by taking the JSON output of
peek
, running whatever algorithms on it outside of Pants as an experiment, and then bringing the useful ones into a plugin
But yeah, requesting
TargetDatas
gives you what you want
We can move those outside of
peek.py
into a more central location if it turns out that various plugins will find it useful
f

flat-zoo-31952

02/24/2022, 4:05 PM
okay i'll take a look
b

bitter-ability-32190

02/24/2022, 4:07 PM
I have a plugin I was hoping to write that outputs the dep as a graph, sounds like
peek
already has this? I gotta try it
If so...
hold on to your butts
Uncaught Invalid DAG structure! Found cycle in node path: src/python/pants/engine/fs.py -> src/python/pants/engine/engine_aware.py -> src/python/pants/engine/fs.py.
ðŸĪ”
👀 1
So that's interesting. The import is under
TYPE_CHECKING
but Pants doesn't see that any different
Which makes sense, but also ðŸ˜Ē
h

hundreds-father-404

02/24/2022, 6:04 PM
yeah Pants can't see it differently, otherwise MyPy wouldn't work
b

bitter-ability-32190

02/24/2022, 6:04 PM
I'll blacklist this dep
(in my script)
Nevermind, there are several 😭
h

hundreds-father-404

02/24/2022, 6:08 PM
yeah, lots of import cycles
f

flat-zoo-31952

02/24/2022, 6:13 PM
you can always write a bunch of `Protocol`s to appease the type checker while eliminating import cycles 😉 ðŸ˜Đ 😰
More seriously though, being able to deal with cycles is probably a requirement for this kinda tool, because repos that need it will likely have cyclical deps
h

hundreds-father-404

02/24/2022, 6:18 PM
Yeah we went very very out of our way in Pants 2.0 to add cycle tolerance. It Was A Thing
"The tool should adapt to you, not you have to adapt to the tool"
b

bitter-ability-32190

02/24/2022, 6:43 PM
The tool this is using for rendering does cool things with DAGs, shame Pants deps have cycles ðŸ˜Ē
pants.html
h

hundreds-father-404

02/24/2022, 6:48 PM
have you seen CoarsenedTargets? It coarsens up cycles to treat them like a single target. Needed for JVM compilation
ðŸĪ” 1
5 Views