https://pantsbuild.org/ logo
Powered by
#general
  • w

    witty-crayon-22786

    04/25/2017, 8:10 PM
    @straight-waiter-86090 : that's where using jars can come in. i discussed this a bit over here: https://contributors.scala-lang.org/t/asking-for-your-feedback-on-sbt-scala-center-announcement/738/20?u=stuhood
  • w

    witty-crayon-22786

    04/25/2017, 8:11 PM
    and the escape hatch (which i mentioned before) is the
    Copy code
    --[no-]compile-zinc-use-classpath-jars (default: False)
    Use jar files on the compile_classpath. Note: Using this option degrades
    incremental compile between targets.
    option... which (mostly) disables incremental compile
  • w

    witty-crayon-22786

    04/25/2017, 8:13 PM
    Copy code
    Finally, achieving incremental compile within jars would be a massive boost as well: ie, compiling directly to output jars, and then supporting incremental compile that consumes the slightly modified jars. The JVM hates classpaths containing loose classes in directories (anecdotally, jarring all inputs improved cold-build performance by 4x in one case due to drastically fewer syscalls and less IO), and this definitely affects Zinc when lots of modules are in play.
  • w

    witty-crayon-22786

    04/25/2017, 8:14 PM
    so if you experiment with the 
    --[no-]compile-zinc-use-classpath-jars
    option, make sure that you also have some useful benchmarks of incremental compile in place.
  • s

    straight-waiter-86090

    04/25/2017, 8:17 PM
    ok, I’ll have to study this.
  • w

    witty-crayon-22786

    04/25/2017, 8:18 PM
    loose directories on classpaths are very expensive... before test runtime, jarring up all the inputs (as we do), can reduce classloading time by an order of magnitude
  • s

    straight-waiter-86090

    04/25/2017, 8:19 PM
    we have this effect where the compilation seems to get slower as the build goes. maybe these are just the most complicated targets running last with the most dependencies. I graphed the compile time per invocation of zinc from a single build:
  • w

    witty-crayon-22786

    04/25/2017, 8:20 PM
    @straight-waiter-86090 : that's why you should use strict_deps.
  • s

    straight-waiter-86090

    04/25/2017, 8:20 PM
    @User uploaded a file:

    Screen Shot 2017-04-25 at 4.18.11 PM.png▾

  • w

    witty-crayon-22786

    04/25/2017, 8:20 PM
    in the absence of strict_deps, the entire transitive graph needs to be on the classpath.
  • w

    witty-crayon-22786

    04/25/2017, 8:20 PM
    and the compiler needs to scan it (or hopefully hit the zinc analysis cache, but that only works for analysis)
  • s

    straight-waiter-86090

    04/25/2017, 8:21 PM
    ok, we’ll try strict_deps further. we tried it in one project, but that was really at the root of the tree, so the effect probably wasn’t as visible.
  • w

    witty-crayon-22786

    04/25/2017, 8:21 PM
    @straight-waiter-86090 : hm, if you mean "root", that's where it should be most noticeable
  • w

    witty-crayon-22786

    04/25/2017, 8:21 PM
    er, "root as in binary/test"
  • w

    witty-crayon-22786

    04/25/2017, 8:22 PM
    because the transitive graph is largest there
  • s

    straight-waiter-86090

    04/25/2017, 8:22 PM
    it was in a project with lots of common code that most other projects depend on.
  • s

    straight-waiter-86090

    04/25/2017, 8:23 PM
    the projects that depend on it don’t have strict_deps yet.
  • w

    witty-crayon-22786

    04/25/2017, 8:23 PM
    not sure i understand... are we using "root" the same way here?
  • w

    witty-crayon-22786

    04/25/2017, 8:23 PM
    if it has any dependents, it's not a root
  • s

    straight-waiter-86090

    04/25/2017, 8:24 PM
    sounds like I’m saying the opposite.
  • w

    witty-crayon-22786

    04/25/2017, 8:24 PM
    yea.
  • w

    witty-crayon-22786

    04/25/2017, 8:24 PM
    the things without dependents are roots: your binaries and tests usually
  • w

    witty-crayon-22786

    04/25/2017, 8:24 PM
    those are the things with the largest transitive dependency graphs, and they're the ones most affected by strict_deps
  • w

    witty-crayon-22786

    04/25/2017, 8:25 PM
    because the classpath goes from 2000 transitive deps to 20 direct deps, for example
  • s

    straight-waiter-86090

    04/25/2017, 8:26 PM
    ok, makes a lot of sense. I’ll try and see how it goes.
  • f

    fast-megabyte-33579

    04/25/2017, 8:54 PM
    strict_deps works great on our java code, but last time we tried it we got a lot of false reports from scala code
  • w

    witty-crayon-22786

    04/25/2017, 8:58 PM
    probably not false so much as just "crappy"
  • w

    witty-crayon-22786

    04/25/2017, 8:59 PM
  • w

    wide-energy-11069

    04/25/2017, 9:13 PM
    does specs2 work in general under pants? which uses junit test runner
  • w

    witty-crayon-22786

    04/25/2017, 9:14 PM
    if you explicitly mark it with the "runwith" annotations, then yes it should.
1...171817191720...2420Latest