#9386 Pants flaky-test policy Issue created by jsirois This issue just serves to document the current Pants flaky test policy. If that policy changes over time, this issue can be updated. # Flaky Test Policy (As of 3/24/2020 as discussed here: https://groups.google.com/d/msg/pants-committers/D-9zkIa9zbQ/KO0pCwf4AAAJ): When a test flakes in CI or the test is already associated with a

flaky-test

issue, feel free to: • If the test is a python test, decorate it with: @pytest.mark.skip(reason="[pointer]") Or: @unittest.skip("[pointer]") • If the test is a rust test rust add: #[ignore] // [pointer] The

[pointer]

can just be

flaky

, but it could also be

flaky: [url]

where the url points to a github issue labelled `flaky-test` if you intend to work on the flaky test or would like to leave extra info about how the test flakes for someone who will pick up the work. If the skipped test has a

flaky-test

issue associated, please leave the issue open but further label it `test-skipped`. The issue can be closed when the test is deleted or fixed and un-skipped. If a test target is flaky due to timeout it's harder to place blame on any individual test in the suite. You may not be in a good position to skip the test, but use your judgement. At the least, please also label any associated issue with `test-timeout`. pantsbuild/pants