I am wondering if it makes sense for us to ignore ...
# general
a
I am wondering if it makes sense for us to ignore
pantsd
, leaving it permanently disabled for all of our builds.
Our build trees are 10M+ files, and 100K+ directories. Our builds take hours. Our builds are kicked off from farm compute, meaning the compute resource running
pants
is temporary and any processes spawned from this will be destroyed upon completion of the build. These three constraints all tell me that
pantsd
is likely not interesting to us, and we can disable it without heartache, because: • Our filesystem inputs won't (shouldn't) change during a build. • Even if they did, so many files/directories are involved that the filesystem-watching capabilities of
pantsd
are strained or defeated. • The
@rule
-caching behavior of
pantsd
is defeated by there "never" being a second
pants
invocation for the owning user, on that host, even if it survived process killing at the end of the farm compute session. • We don't care about saving several seconds of startup time, among hours of build work, even if we were able to leverage it. (And there isn't much more time to save here, as there's functionally no "build graph" for Pants to build at time zero, for our setup). • We do massively benefit from the other caching mechanisms (e.g. process result caching during remote execution), on which we lean much of our potential benefit from Pants. Does this claim make sense, that
pantsd
isn't buying us much and we can disable it without heartache? Or am I missing something critical? We have been building our Pants story without it up to this point, so this is a sanity check that we can comfortably ignore it going forward. (And I know these are atypical constraints and that
pantsd
is usually a huge boon.)
w
🎯
a
Excellent. Thank you!
w
Now, a caveat is "it depends" - but I don't think I run pants in CI at all. There are cases where it can work, like if I was self-hosted and wanted a warm memoization or something - but to your point, your build is ridiculously large
a
Of course. We'll continue to keep it in mind where it might help us, but I wanted to sanity check that I wasn't missing something obvious for our typical case. Thank you!!
w
You'll still get file caching, I believe, as I think that's a separate flag vs
no-pantsd
I use a slightly different tool I built for introspection, which is much faster, but also much less capable - and even it would fall over on your repo
a
Yes. The other caching mechanisms have all been helpful/critical.
🔥 1
h
I agree, pantsd is giving you next to nothing here, and at no small cost