Also, need a shipit on this cherry-pick that had a merge conflict: https://github.com/pantsbuild/pants/pull/23676