https://pantsbuild.org/ logo
Join Slack
Powered by
is there a way of controlling amount of memory use...
# general
w
is there a way of controlling amount of memory used by nailgun in JVM processes? or limiting the number of the nailgun servers spawned in some way or another? We are experiencing some crashes when trying to build our codebase (when trying to build a large number of JVM targets). In Linux machines the error is:
Copy code
Exception: Error communicating with server: Client exited before the server's result could be returned.
Which makes me think that the nailgun server process was killed/crashed. In MacOS the result is more dramatic, after a while you start getting warnings from the OS saying that you are out of memory leading to the whole computer crashing. The only way we have found now to control this is by using 
[GLOBAL].process_execution_local_parallelism
, which is a blunt hammer as it affects all processes.
f
Not that I’m tacitly aware of. Pants probably needs an option to allow passing JVM options to the JVM running the nailgun servers. You would then be able able to pass JVM heap size options to control the amount of memory usage.
cc @witty-crayon-22786 who knows more about the nailgun logic in Pants than me
w
I think it would need a two front approach: • being able to pass JVM options to the nailgun JVM • Pants having a way of controlling the spawning of nailgun instances, like a pool or something (if not already in place)
we noticed that Pants was spawning a new nailgun server per local parallel process
that’s how we managed to tame it down using the 
[GLOBAL].process_execution_local_parallelism
setting
w
But yes: there is already a pool of nailgun servers. We spawn 2x the parallelism.
Note though that if pants is `KILL`d rather than `TERM`d, you can hit https://github.com/pantsbuild/pants/issues/12996, which can orphan nailgun servers, and make it look like we spawn more than 2x
w
ah, ok, that could explain the swarm of nailgun processes I was seeing sometimes when simply killing pantsd from a terminal
could there be other instances in which the pantsd process dies leaving those other processes behind?
w
There shouldn't be, but potentially.
w
would be interesting to test this again once the JVM memory controls are in place
for now I think we can live with the limit on number of parallel processes as a proxy setting
w
Both of these are important polish issues that we want to dive into in the next two months or so. The memory controls one shouldn't need to dive too deep into the engine, but it will definitely require a bit more design.
👍 1
I'd be happy to sketch more on the ticket if you were interested in working on it!
w
sure, this is something that has bothered me a bit in the last week or so, so wouldn’t mind having a stab at it
w
awesome. yea, i’ll sketch that out before your tomorrow.
i expanded the description of https://github.com/pantsbuild/pants/issues/13067 quite a bit… it has roughly the same shape as before, but more details.
no pressure if you don’t feel like tackling it though.
w
just seen it now as I’m having quite a busy day but could start doing something either tomorrow or next week.
w
yea, no worries at all
3 Views
Open in Slack
PreviousNext
Powered by