Visibility rules incur high overheads with a very large monorepo. Just disabling it reduces the execution time of

pants lint ::

from 6 min to 30 seconds (without pantsd). I’m doing some analysis with AI agents to figure out what is causing this. I’ll share some results soon.

Thanks for digging into this!

in that case, running the same command right after the first run reuse the memoized results, which finishes instantly. is there any way to avoid that to test the dependency resolution caching only?

Hmm, good question. I wonder if

pants peek ::

can act as a proxy for this?

This is fascinating since I just today did some profiling that came to the opposite conclusion (at least on

main

). Looking forward to comparing notes!

Well, this is yet more interesting, because I did some static review a couple of weeks ago, and thought I noticed that visibility rules ran "later" than they should for perf savings, but didn't notice ANY performance difference in one repo I tested in 😆 😢

Now I've gone back to thinking they do have meaningful overhead, but I'm even less sure how to measure properly. Looking forward to your notes @rhythmic-morning-87313

The “benchmark” could be done using

time pants peek ::

on a working copy of

<https://github.com/lablup/backend.ai>

(as it’s a quite big repo, i recommend using

--fetch-depth=10

to limit the download size). I have some interesting benchmark results: • DGX Spark (based on Ubuntu 24.04): ~80 seconds • AMD Ryzen 9950X3D2 (Windows 11 Pro 24H2 WSL2 Ubuntu 26.04): 4 ~ 10 minutes (!) • AMD Ryzen 9950X3D2 (native Ubuntu 26.04): ~35 seconds • Apple M4 Pro (MBP16, OrbStack VM Ubuntu 24.04): ~50 seconds • Apple M4 Pro (MBP16, native macOS 26.5): ~50 seconds

Oh interesting, is

<http://backend.ai|backend.ai>

"your" repo, or just a handy public one?

It’s “my” repo.

“mapping dependency targets”

after repeating several runs (with pantsd), there seems to be no big difference between pants v2.29 and 2.32

So the original issue was

pants lint ::

was taking ~6 minutes with pantsd already warm? Please feel free to make an issue in the style of https://github.com/pantsbuild/pants/issues/23236 or https://github.com/pantsbuild/pants/issues/18911 I know several people are interested in adjacent performance cases.