On 2.14.1. I've been seeing the following happenin...
# general
h
On 2.14.1. I've been seeing the following happening recently where the scheduler repeatedly gets initialized. Anything I should be looking for?
Copy code
15:45:27.60 [INFO] Initializing scheduler...
15:45:27.83 [INFO] Scheduler initialized.
15:45:43.12 [INFO] Initializing scheduler...
15:45:43.50 [INFO] Scheduler initialized.
w
should look at
.pants.d/pants.log
to see why
pantsd
is restarting
h
ah, yep. memory limit exceeded
Copy code
15:46:16.86 [ERROR] The scheduler was invalidated: Exception('pantsd process 3162557 was using 1027.84 MiB of memory (above the `--pantsd-max-memory-usage` limit of 1024.00 MiB).')
15:46:17.35 [ERROR] service failure for <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7efdb00b90d0>.
15:46:17.36 [INFO] Waiting for ongoing runs to complete before exiting...
15:47:21.80 [INFO] Extending leases
15:47:27.39 [INFO] Done extending leases
Is it pretty common to bump that up? Already seems like a pretty conservative buffer.
That's after running
./pants test <a lot of individually specified targets>
. The list of targets is automatically generated based on some local settings to smoke test what changed locally down a couple dependency levels.
w
Is it pretty common to bump that up? Already seems like a pretty conservative buffer.
it depends… if the usage feels out of line for your use case, please definitely open an issue (including the output of
--stats-memory-summary
would be good).
h
I see this a lot in our internal CI. I think we should probably bump the default to 2GB.