I think there is an argument for not caching failed integration tests, although it is an open question where that logic rightly could live

Agreed. See https://github.com/pantsbuild/pants/issues/8292#issuecomment-531715202. Comments appreciated as this is a new API we only recently started thinking about / haven’t finalized, so feedback is very leveraged