@happy-kitchen-89482 I'm I remembering correctly that you had a local (NG?) branch where you had used the dep batching from https://github.com/pantsbuild/pants/pull/23075 ? I had played around with trying to batch by directory at one point, and https://github.com/pantsbuild/pants/pull/23434/changes has few "mega-batches". I'm trying to figure out how to coordinate the right approach

Yes, NG uses this, I'll find a link. I am actually looking into how to use #23075 to speed up

dependents

computation even in the existing backend. It's not trivial because the entire dep inference edifice expects to operate on a single target at a time.

I thought @worried-painter-31382 also had a PR for something like this?

Mine is a deeper change, allowing rules and cache to be implemented in rust and therefore gil free

So is this just assuming everything is python and ignoring other dep inference implementations? How would it work with custom dep inference?

If 'this' is 23434, that's @gentle-flower-25372 . If `this was my "dir at a time" branch... I have not managed to find it again yet ;-/

OK sorry, I got the wrong end of the stick here, I thought this was your PR. @gentle-flower-25372 and I have been chatting about this problem for a while, so I will follow up with him.