My pants deamon is dying frequently with the follo...
# general
e
My pants deamon is dying frequently with the following message. Every time I run the package goal and others.
Copy code
23:31:28.31 [ERROR] saw filesystem changes covered by invalidation globs: content changed (Digest('d4cd7349f655cdec6b7205d5639dc60a2c0bb7b23c9c63e60abc9ac129b3abae', 337) fs Digest('c02add6f83006535b9c4d0382501f86a486283419549edd02e97906b04d0a28a', 337)). terminating the daemon.
23:31:28.49 [ERROR] service failure for <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7f498aef4fd0>.
Any tips on figuring out what files are actually causing the digest to change? In the case where just the digest and not the fileset changes it doesn't look like the culprits aren't even calculated: https://github.com/pantsbuild/pants/blob/e70071244c20cd377c88bcbf87c4a6b0ec7d68b4/src/python/pants/pantsd/service/scheduler_service.py#L109 It looks like the invalidation glob could potentially be large: https://github.com/pantsbuild/pants/blob/e70071244c20cd377c88bcbf87c4a6b0ec7d68b4/src/python/pants/option/global_options.py#L1540
h
Are you setting custom
invalidation_globs
in your pants.toml?
And have you looked in
.pants.d/pants.log
?
You should see that ERROR but before it there should be INFO logs showing which files changed
e
Ahh I see the "notify invalidation" messages now. They are for docker and python files in one of my source roots. Is this intentional?
Figured it out! Also explains why I only sometimes see this behavior (pantsd restarting all the time). Our repo's legacy env setup script sets
$PYTHONPATH
and sometimes I run
./pants
from inside a terminal where that virtualenv has been activated. This makes my entire source tree part of the invalidation glob!
h
Ooooh, painful...
Glad you figured it out!
Hmm, can you explain further? I didn't think we looked at the external PYTHONPATH at all
w
the
pants
executable won’t, but the
pants
script might …? still a bug obviously, but…
h
Yeah but that shouldn't affect the pantsd invalidation globs
w
pantsd
defaults to invalidating based on its own
sys.path
e
Must be something from the env...
w
That's a pants-specific option `--pythonpath`: it's only used for plugin development, so you shouldn't need to set it unless you are developing plugins
e
Is there any way to see the full invalidation glob tuple? My
src
directory is ending up in there somehow.
Not setting it in my toml.
w