what do people think about the possibility of a `pants prune

Question

what do people think about the possibility of a `pants prune` goal

hundreds-father-404 · Answer

what does it do

hundreds-breakfast-49010 · Answer

or `pants clean` or some other goal that tries to manually determine what artifacts on disk pants has left around and clean them

hundreds-father-404 · Answer

I d like it Even though pantsd is supposed to do garbage collection Asher has pointed out it doesn t work very well Pants takes a ton of storage if you don t manually prune and it s very non obvious imo to users that they can should delete things

hundreds-father-404 · Answer

Can we prune `lmdb store` intelligently like only get rid of unused stale things

fast-nail-55400 · Answer

and as prior art many other tools have a manual prune mode

hundreds-breakfast-49010 · Answer

docker for one

hundreds-father-404 · Answer

I think I like the name `prune` more than `clean`

fast-nail-55400 · Answer

yup `docker system prune`

fast-nail-55400 · Answer

and `git gc` of course

hundreds-breakfast-49010 · Answer

I don t know how hard it would be to manually figure out what we can safely deletein lmdb store

hundreds-father-404 · Answer

Especially if we could have options to completely get rid of everything vs to only remove stale things etc

fast-nail-55400 · Answer

you d need to define some set of roots

fast-nail-55400 · Answer

and then prune anything that is not reachable from those roots

hundreds-breakfast-49010 · Answer

part of the problem is that pants stores its on disk artifacts in a couple of different places which I d like to unify in any case

hundreds-father-404 · Answer

I think the Pantsd garbage collection is supposed to already prune intelligently But I have no idea how it works Stu would know

hundreds-breakfast-49010 · Answer

off the top of my head ` cache pants` and ` lt build root gt pants d` not all of which is actually pantsd related code anyway

hundreds-breakfast-49010 · Answer

I may be missing some things

jolly-midnight-72759 · Answer

We call our shell script that does this `build suppoet nuke` mushroom cloud

jolly-midnight-72759 · Answer

Unfortunately it uses `pants` to find the proper directories to get rid of so if you run it twice it rebuilds `pants` chroot before deleting it rolling on the floor laughing

hundreds-breakfast-49010 · Answer

curious what your shell script does if you can share it < jolly midnight 72759>

jolly-midnight-72759 · Answer

There are better ways to do this I m sure but we didn t want parse `pants toml` until we deployed v2 0 0 If this gets build into `pants` proper that would be even better

jolly-midnight-72759 · Answer

oh sharing a code snippet in a thread is not a happy slack place

fast-nail-55400 · Answer

on calling things nuke the toolchain internal repo CI config has a shell function called `nuke if too big` to prune Pants caches

jolly-midnight-72759 · Answer

four letters a hard sound and a cromulent definition makes it apropos

hundreds-breakfast-49010 · Answer

clearly this is something that multiple people are running into and building their own tools to solve

hundreds-father-404 · Answer

We document `nuke if too big` in our Using Pants in CI page too

hundreds-breakfast-49010 · Answer

the log stuff I m about to add is going to make the files we store in ` pants d run tracker` bigger which is why I m thinking about this

polite-garden-50641 · Answer

I think asking the user to run that is bad UX it should just happen automatically based on some sane defaults just like git gc

polite-garden-50641 · Answer

it is fine to have a way to manually run it but it should happen automatically by default

hundreds-breakfast-49010 · Answer

agreed but it makes sense to build a manual feature before we try to make it automatic and in the meantime it can solve problems users are actually encountering

polite-garden-50641 · Answer

I agree but I don t think it is a priority right now pants already writes a lot of stuff without cleaning it up so this would so a directory with a bunch of log files won t be anything new or the biggest space hog

hundreds-breakfast-49010 · Answer

yeah I don t think the log stuff I m about to add will significantly increase the amount of on disk space pants uses but it will increase it a little bit

hundreds-breakfast-49010 · Answer

and in general it is already a problem that pants disk usage can get huge we ve all run into this

hundreds-breakfast-49010 · Answer

that is a problem worht solving or at least mitigating to make the post 2 0 0 user experience better

hundreds-father-404 · Answer

gt and in general it is already a problem that pants disk usage can get huge we ve all run into this +1 And I don t think it s a good experience for a user to realize how hungry Pants is and not having a mechanism to clean it without knowing it s safe to `rm rf` +1 that automatic is better but manual is a fine first step

polite-garden-50641 · Answer

I doubt those logs can get any where near the tens of gigs unless debug logging is turned on by default which it shoudn t

hundreds-breakfast-49010 · Answer

logs are not the problem they just caused me to think about the actual problem which is everything we store in ` cache pants ` particularly lmdb and rust artifacts