For JVM languages, I wonder why pants chose to produce jar files and not class files? I think some performance will be gained by not zipping the class files, stripping the jar file and then for the scalac/javac to uncompress the jars. Plus I think if Pants will eventually want to batch the compilation phase, splitting the class files in a digest might be easier than splitting a jar file.
I tried that locally and the switch was pretty simple, numbers look too good at the moment (50%) so I try to figure out the real improvement (I changed the cache scope of scalac to PerSession to test the improvement).