https://pantsbuild.org/ logo
Powered by
#general
  • w

    witty-crayon-22786

    04/25/2017, 5:47 PM
    not a good one (a patch to improve it would be appreciated):
  • w

    witty-crayon-22786

    04/25/2017, 5:47 PM
    --batch-size=1
  • w

    witty-crayon-22786

    04/25/2017, 5:48 PM
    the issue is that it's batching at the class level... and scala generates a lot of classes.
  • w

    witty-crayon-22786

    04/25/2017, 5:48 PM
    ...many of which under a test target are not tests themselves.
  • w

    witty-crayon-22786

    04/25/2017, 5:49 PM
    in CI, we do the equivalent of: 
    ./pants minimize :: | xargs -L1 ./pants test
  • w

    witty-crayon-22786

    04/25/2017, 5:49 PM
    (...across many machines.)
  • r

    rich-helicopter-79006

    04/25/2017, 5:49 PM
    Hmm. Never heard of 
    pants minimize
  • w

    witty-crayon-22786

    04/25/2017, 5:49 PM
    but yea... a patch to explicitly fork... per target? would be handy.
  • r

    rich-helicopter-79006

    04/25/2017, 5:50 PM
    Yeah - totally understand. I just started working with pants, so maybe I can take a look sometime
  • r

    rich-helicopter-79006

    04/25/2017, 5:50 PM
    Also, what is the interaction between default test concurrency and the number of parallel threads?
  • r

    rich-helicopter-79006

    04/25/2017, 5:51 PM
    I assume if the number of parallel threads is 
    1
    then no matter the concurrency the tests are executed serially, correct?
  • w

    witty-crayon-22786

    04/25/2017, 5:51 PM
    correct
  • r

    rich-helicopter-79006

    04/25/2017, 5:53 PM
    TY
  • s

    straight-waiter-86090

    04/25/2017, 7:17 PM
    is there a way to skip zinc analysis if there is a small number of sources to compile for a target? I see recompile-all-fraction as an option, but that still does analysis.
  • w

    witty-crayon-22786

    04/25/2017, 7:25 PM
    hm... to not record it at all? not really, no.
  • w

    witty-crayon-22786

    04/25/2017, 7:25 PM
    are you seeing a lot of overhead for small targets?
  • s

    straight-waiter-86090

    04/25/2017, 7:27 PM
    that’s a theory I have. we have a lot of packages with a very small number of files (like 1 or 2). Running yourkit against nailgun I see a lot of time spent in scala code.
  • s

    straight-waiter-86090

    04/25/2017, 7:27 PM
    wondering if that analysis time is higher than just compiling for small packages.
  • w

    witty-crayon-22786

    04/25/2017, 7:28 PM
    i've seen up to 50% overhead in some cases i've tried, yea.
  • w

    witty-crayon-22786

    04/25/2017, 7:29 PM
    one of the issues with not having analysis for a target though is that it will cause full recompilation of things above it in the graph
  • w

    witty-crayon-22786

    04/25/2017, 7:29 PM
    so if that tiny target has big dependents, they will be fully invalidated
  • w

    witty-crayon-22786

    04/25/2017, 7:30 PM
    we're definitely interested in lowering the overheads there. a zinc upgrade was in for a few 1.3.0.dev releases, but it's reverted now in master because of https://github.com/pantsbuild/pants/issues/4477
  • s

    straight-waiter-86090

    04/25/2017, 7:32 PM
    I see. I guess for leaves in the graph that have no dependencies it might make sense, but I see what you means for anything that does have dependencies.
  • s

    straight-waiter-86090

    04/25/2017, 7:32 PM
    I’ve seen this flag mentioned but I can’t find any reference to it in the source: 
    '-Dzinc.analysis.cache.limit=1000
    . What does it do and is there a way to tell if this limit is the right size or too small?
  • f

    fancy-queen-20734

    04/25/2017, 7:55 PM
    It’s the in-process cache for zinc for analyses
  • a

    adventurous-pharmacist-28373

    04/25/2017, 7:59 PM
    Is there currently a way to build python projects into AWS lambda compatible artifacts? Or would this have to be done outside of the generated pex file?
  • w

    witty-crayon-22786

    04/25/2017, 8:04 PM
    @straight-waiter-86090 : the size of a weakreference cache of analysis files for targets: https://github.com/pantsbuild/pants/blob/master/src/scala/org/pantsbuild/zinc/Setup.scala#L179
  • w

    witty-crayon-22786

    04/25/2017, 8:04 PM
    if you miss the analysis cache for a dependency, you'll have to load it from disk
  • s

    straight-waiter-86090

    04/25/2017, 8:07 PM
    ok. thanks. my yourkit profile showed significant time spent in File.exists and File.isDirectory. I was wondering if there was a way to reduce that.
  • s

    straight-waiter-86090

    04/25/2017, 8:07 PM
    @User uploaded a file:

    Screen Shot 2017-04-25 at 4.05.50 PM.png▾

1...171717181719...2420Latest