@freezing-motorcycle-14222: tests/python/pants_test/backend/jvm/tasks/test_benchmark_run.py has a single test in it from the last time bench broke, you could add something more comprehensive there.