Hi guys, I hope you can help me with the issue we’...
# general
Hi guys, I hope you can help me with the issue we’ve faced :) We use pants 2.7.0 for our mono repo with about 20+ projects and since we switched from managed Github Actions runners to self-hosted we notice that pants is consuming a lot of memory to run python_tests. When the limit was 4Gib the pods were often killed. Since then we’ve increased the limit to 8Gib and now it looks pretty stable, but sometimes the consumption goes as high as 85%=~7Gib and as the codebase is going to grow together with the tests, I have some concerns. I believe we face a combination of https://github.com/pantsbuild/pants/issues/12662 and https://github.com/kubernetes/kubernetes/issues/43916 here, and there are a few workarounds around this issue, but maybe I’m missing/misconfigured something on the pants side as well. I would appreciate any help :)
CC @better-cricket-91243 🙂
🙌 1
Here’s pants configuration we use for CI:
Copy code
pantsd = false

process_execution_local_cache = false

remote_cache_read = true
remote_cache_write = true
remote_store_address = "<grpc://some.cache.domain>"

dynamic_ui = false
colors = true

process_execution_local_parallelism = 4

resolver_jobs = 1

use_coverage = true

log = true
as the person assigned that ticket, i’d be remiss not to respond! heh
frankly, we don’t have any tools in place for this, so the best bet is going to be a combination of native and python memory profiling tools
👍 1
and that is worth trying, in case you see any smoking guns.
to tackle it more deeply though, i expect that we’ll need to add support to pants for reporting the top-memory-consuming Graph nodes, followed by some optimization of whatever that ends up showing.
👍 1