https://pantsbuild.org/ logo
Join Slack
Powered by
I'm trying to brainstorm to leverage Pants' dep in...
# general
f
I'm trying to brainstorm to leverage Pants' dep inference + explicit dep modelling, and I'm wondering if it's possible to run some graph algorithm type stuff on this info? An example: to find the betweenness-centrality to find the most central dependencies. Is there a target type that I could 
Get
to use this as an input to this calculation?
b
FWIW I'm probably going to author (and maybe publish) a plugin that'll give you the dep graph for the entire repo so you could do various calculations on it
👍ðŸŧ 1
f
Yeah I think that's what I'd need. There doesn't seem to be something in between 
TransitiveTargetsRequest
and 
DependenciesRequest
which implies you'd just need to recurse and build that graph up
b
Yeah. I don't think there's anything in-Pants to do this yet, but should be relatively trivial to author
h
If you look at the implementation of the 
peek
goal, it pretty much generates a graph data structure
👀 1
f
Wait didn't I write that?
ðŸĪĢ 1
Have I already implemented this and just forgot?
h
f
h
You did!
I changed it some, but you basically did
haha
f
it's been a long time and I have a short memory 😅
ðŸĪŠ 1
h
You could start by taking the JSON output of 
peek
, running whatever algorithms on it outside of Pants as an experiment, and then bringing the useful ones into a plugin
But yeah, requesting 
TargetDatas
gives you what you want
We can move those outside of 
peek.py
into a more central location if it turns out that various plugins will find it useful
f
okay i'll take a look
b
I have a plugin I was hoping to write that outputs the dep as a graph, sounds like 
peek
already has this? I gotta try it
If so...
hold on to your butts
Uncaught Invalid DAG structure! Found cycle in node path: src/python/pants/engine/fs.py -> src/python/pants/engine/engine_aware.py -> src/python/pants/engine/fs.py.
ðŸĪ”
👀 1
So that's interesting. The import is under 
TYPE_CHECKING
but Pants doesn't see that any different
Which makes sense, but also ðŸ˜Ē
h
yeah Pants can't see it differently, otherwise MyPy wouldn't work
b
I'll blacklist this dep
(in my script)
Nevermind, there are several 😭
h
yeah, lots of import cycles
f
you can always write a bunch of `Protocol`s to appease the type checker while eliminating import cycles 😉 ðŸ˜Đ 😰
More seriously though, being able to deal with cycles is probably a requirement for this kinda tool, because repos that need it will likely have cyclical deps
h
Yeah we went very very out of our way in Pants 2.0 to add cycle tolerance. It Was A Thing
"The tool should adapt to you, not you have to adapt to the tool"
b
The tool this is using for rendering does cool things with DAGs, shame Pants deps have cycles ðŸ˜Ē
pants.html
pants.html
h
have you seen CoarsenedTargets? It coarsens up cycles to treat them like a single target. Needed for JVM compilation
ðŸĪ” 1
5 Views
Open in Slack
PreviousNext
Powered by