Hello fellow pants-wearers! I'm wondering where I ...
# general
s
Hello fellow pants-wearers! I'm wondering where I can find documentation on
pantsd
, what it does and is responsible for. The only docs I can find are from pants v1.
e
Not sure about docs, but at a high level, its basically just a running process with an in-memory cache so you can get results even faster than the filesystem cache. You can turn it off with configs: https://www.pantsbuild.org/stable/reference/global-options#pantsd and it will impact performance, but not correctness.
h
Huh, I assumed we had explicit documentation about the daemon, but it looks like we don’t. Probably because `pantsd`*is* pants at this point.
But it exists for two main reasons: 1) To keep build state cached in memory. In particular this means the results of in-process rules (the results of subprocesses are cached on disk). 2) To watch the filesystem for changes, so it can invalidate efficiently.
So in normal use
pants
is just a thin client that delegates work to
pantsd
. You can turn
pantsd
off, in which case all the work runs in that client, and you get no rule caching or filesystem watching
w
FWIW, I'd like to see us get to the point where we're fast enough that
pantsd
is a "oh thats nice" rather than "goddamn I need this"
s
Thanks all 🙂 I'm working in a huge (275,000 files, 30-50 services) monorepo, and we're fighting resource constraints here and there. I find
pantsd
regularly eating some 4-6 gigs of RAM and
~/.cache/pants/
regularly gets bigger than 30-40GB. I'm mostly just curious to learn more about why.
w
You can get a slightly better idea of what’s going on with the stats subsystem: https://www.pantsbuild.org/stable/reference/subsystems/stats#memory_summary But yeah, usually what you’d find is cache on cache on cache - other than the downloaded and prepared libraries/dependencies/etc in your disk cache - and as you modify versions, that continually grows. I believe there is or will be some updates to pex that allows disk cleanup - but I haven’t looked into that deeply