https://github.com/pantsbuild/pants/pull/19099 appears to stabilize (and speed up the wall time of) CI. Once I merge it (review plz!) I highly recommend merging it into your PR branches.