For the record: https://github.com/pantsbuild/pants/issues/4800 I'm experimenting with this and will update that issue so watch there if you're interested.