#13505 Add generic test retry support Issue created by Eric-Arellano This gives a generic mechanism for languages that might not have support out of the box. Something like this:

python_test(name="py", source="tests.py", retries=3)
shunit2_test(name="sh", source="tests.sh", retries=2)
go_package(name="go", test_retries=4)

A cool benefit is being able to use ./pants peek to analyze which tests are being retried. However, a limitation is that you can't retry individual tests. You have to retry the whole thing. For more granular retries, you must use mechanisms from your language. -- The implementation should be pretty easy: run the

Process

in a

for

loop over

range(# num retries)

until success or exhausting the loop. The tricky part is figuring out what to display to the user. For example, do we show the std{out,err} of the failures? Unlike #8528, I do not think we need to have any type of auto-tagging system for flaky tests. All Pants needs to expose is a

retries: int

field, and leave it up to users to combine that with

./pants peek

to manage flaky tests over time. (We could always add some type of more automated system over time.) pantsbuild/pants