<#22393 Performance Issue: `pants paths` consumes ...
# github-notifications
c
#22393 Performance Issue: `pants paths` consumes excessive memory on large repos Issue created by jasonwbarnett Describe the bug Running
pants paths
on our large monorepo consumes an excessive amount of memory and runs for 10+ minutes without completing. This happens even after filtering to
python_source
targets. At its peak, the process consumed ~650 GB of RAM on an
r7i.metal-48xl
instance (1536 GiB total). Repo stats: • ~3,400 source targets • ~16,000 destination targets • After adding `--filter-target-type=python_source`: • ~2,000 source targets • ~6,600 destination targets Despite that, performance remains extremely slow. There appears to be either a performance bottleneck in the graphing logic or a memory inefficiency. According to @benjy, the current Python-based graph algorithm is likely the culprit, and could benefit from a rewrite in Rust with more efficient data structures. Pants version 2.26.0 OS Amazon Linux 2023 Additional info Happy to provide more logs or run experiments. This was blocking me from visualizing connections between resolves, which would've helped validate our target structure. Any flags or workarounds to scope this down further would also be appreciated. pantsbuild/pants