Tangentially related the above, what do people thi...
# development
b
Tangentially related the above, what do people think about dynamic config options? With devs having varying machinery it's hard for me to set, for example the
pantsd
memory size, in
pants.toml
for fear I'll lightly nuke someone's laptop. being able to set it to a %-of-resource could help. As a datapoint, I kept seeing
pantsd
restart on each
lint
run because the default wasn't enough. I don't expect the data scientists who run these commands to notice it's even an issue, let alone know what to set and to what.
f
Are you viewing logs to become aware of those restarts?
(and/or processing logs)
to your question, Iā€™d be fine with a dynamic config if there were a way to compute a good dynamic default.
but that really depends on the particular config setting
b
Yes to logs (which I think you pointed me to šŸ˜‰ )
But I only was made aware when I noticed certain rules running I wasn't expecting to be running (due to memoization)
f
would it help for us to add a counter metric for pantsd restarts?
āœ… 1
then you could
--stats-log
and parse for the counter
but on your question, I wonder how contribution individual workloads will have to what a sensible default is
could be too much variance among users to come up with a sensible dynamic value
maybe better to just let users detect the issue easily and then they just tune for their own usage
b
I think more things logged in stats log is a good idea
f
this is something you could set dynamically in
./pants
with
export PANTS_PANTSD_MAX_MEMORY_USAGE=...
b
Setting it isn't hard, determining it is šŸ˜‚
But also, setting it to, say a percentage of memory avoids both issues in one šŸ˜‰