tests/python/pants_test/pantsd/pantsd_integration_test.py is a real problem, it fails frequently, but not consistently, in CI. And it doesn't appear to be related to system CPU load. E.g., here every other test is cached, and this one still fails.

it's been a problem the past few years. This is not at all solving the problem, but maybe it would be slightly less offensive if we split it up into multiple files so that it's not as slow to restart?

I've bumped the timeouts in the test substantially, and so this may be exposing an actual issue rather than just the test being poorly constructed.

This is the one I've had fail on me the most as well, and it sends me down a spiral of trying to debug where I went wrong - and then realizing I first should try re-starting. Is there a pattern of what the failures are, or some subset of tests that cause the failure?

Is there a pattern of what the failures are, or some subset of tests that cause the failure?

I’m unaware of any tests causing this, but as soon as I see pantsd integration test failures (and no other fails) my first stop is a re-run to see if it was a flake.