I'm still struggling with Pants restarting the scheduler whenever any file changes across all of the machines in my developer team. It seems like all of my source roots are being appended to the Pants invalidation globs and file changes results in an exception that kills the scheduler. My configuration file is not modifying the pythonpath or the pants sys.path directly. Can anyone think of a way to troubleshoot this issue?

Often the pantsd log file will have information about which paths are causing the invalidation, so one step would be identifying some exact paths that are causing this. If you have any in-repo plugins, understanding their dependencies will be relevant too.

We do not have any in-repo plugins. The pants.d logs show the following two messages in order whenever a file is edited:

10:09:04.00 [ERROR] saw filesystem changes covered by invalidation globs: content changed (Digest('e43cc803ec7b0f7f52a9e9f9bfb93d8dc27120e6d3fe25b2f3d69f1108b3d54f', 170) fs Digest('82c2fe5500aea967af7a1d6d9996b7e64335b998edbb122d376808b2b2cd35ac', 170)). terminating the daemon.
...
10:09:04.65 [ERROR] service failure for <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7bdff57c7970>.

I haven't been able to map this digest back to a specific filename, but the digest changes on every run.

Huh, that is unhelpful. Does increasing the log level to debug (https://www.pantsbuild.org/2.20/reference/global-options#level) provide any more info about that?

Ugh. It's difficult to say. Every time I change logging level the whole Pants daemon goes haywire and plays by its own rules for subsequent runs. It did log some additional information when I set the level to trace, but unfortunately when I tried to set it back to

info

it just stopped showing logs entirely, even when restarting the scheduler. Running a single unit test takes ~5 seconds, so I'm assuming it's still restarting the scheduler behind the scenes, and just not printing it.

Ugh! sorry for the trouble. What version of pants?

2.17.0. I'll try upgrading to the latest version now, shouldn't cause any issues 🙂 Thanks for the patience!

To definitely force a restart of the daemon, either

pkill -f pantsd

or

rm -rf .pants.d/pids

should do the trick.

Hmm. Is there a reason some messages aren't appearing in

pants.log

? I saw the

[INFO] Initializing scheduler...

in stdout, but it's not shown in any of the run_tracker logs or the main log file

pants.log

in

.pants.d

Yeah, I think the

pants.log

file is only messages from the daemon itself. I believe some of the messages visible in the terminal are from the client (that talks to the daemon), but and so aren't written to the file.

Ah, okay that makes sense.