https://pantsbuild.org/ logo
Join Slack
Powered by
I'm trying to use `pants paths` between two root d...
# general
g
I'm trying to use 
pants paths
between two root directories in our monorepo to see how they're connected and it's on the struggle bus. I deployed a r7i.metal-48xl instance type to see if I could actually get there. The process is 10m in and memory consumed is up to 42% ~650GB. I'm just trying to get a sanity check. Is that possible or is there a memory leak?
h
How big is the repo?
In # of files?
g
Copy code
Found 3427 source targets and 16157 destination targets
I added this:
Copy code
--filter-target-type=python_source
Dropped it to:
Copy code
Found 2013 source targets and 6613 destination targets
but it still seems really slow for some reason
Is there a way to keep it within a single resolve?
h
I think you’re suffering from the slowness and memory-greed of the python graph algorithm implementations
they really should be rewritten to use much smaller data structures, and in Rust
g
Gotcha.
I'll file an issue.
w
Let's just re-write it all in Rust. Save some time by spending a lot of time 🧠
🤣 1
g
I seriously can't wait until AI models are good enough where re-writes for even complicated code bases is nearly "free" -- We're not that far away from that being reality.
w
As someone who has tried to get AIs to do something in Rust that I can do in about 10 minutes (as a mid Rust developer), I don't share that sentiment 😆
g
Like I said, the models just aren't good enough, yet. They will be eventually.
As powerful as they are for certain tasks, they're still awful.
4 Views
Open in Slack
PreviousNext
Powered by