https://pantsbuild.org/ logo
Powered by
#general
  • w

    wide-energy-11069

    09/08/2015, 8:09 PM
    Hi, I tried to enable appending multiple pants config. https://github.com/pantsbuild/pants/compare/master...wisechengyi:append-overrides Test passes locally but fails on CI because bootstrap_option_values.config_override is expected to be a list but shows up as a string. As a result, full_configpaths has: CI: (u'/home/travis/build/wisechengyi/pants/pants.ini', 'p', 'a', 'n', 't', 's', '.', 't', 'r', 'a', 'v', 'i', 's', '-', 'c', 'i', '.', 'i', 'n', 'i') local: [u'/Users/yic/workspace/pants/pants.ini', 'pants.travis-ci.ini'] Any insight why the discrepancy happens? Tried maybe_list to wrap around but there is other issues later, so it's better to nail down the root cause of this discrepancy.
  • b

    bored-art-40741

    09/08/2015, 8:12 PM
    Oh yeah, this thing wants WAY more memory than I was giving it
  • b

    bored-art-40741

    09/08/2015, 8:13 PM
    And it seems like it wants to classload way more than it was able to
  • b

    bored-art-40741

    09/08/2015, 8:14 PM
    @witty-crayon-22786: I'm getting a lot of perf complaints because of the register_vts thing
  • b

    bored-art-40741

    09/08/2015, 8:14 PM
    Is it plausible that an unclean workspace causes register_vts to do a lot more work than before?
  • b

    bored-art-40741

    09/08/2015, 8:16 PM
    Example:
  • b

    bored-art-40741

    09/08/2015, 8:16 PM
    Copy code
    15:58:49 00:08       [jvm-compilers]
15:58:49 00:08         [apt-pre]
15:58:49 00:08         [java-pre]
15:58:49 00:08         [zinc-pre]
15:58:52 00:11         [zinc-prepare]
15:58:55 00:14           [find-deleted-sources]
15:58:55 00:14           [prepare-analysis]
15:59:01 00:20         [zinc-execute]
15:59:02 00:21           [cache] 
                       No cached artifacts for 1 target.
                       Invalidated 1 target.
16:05:25 06:44           [partition-analysis]
                       Compiling 27 zinc sources in 1 target (partition 1 of 1).
16:06:22 07:41           [compile]
                         
16:06:22 07:41             [zinc]
                           [info] Compile success at Sep 8, 2015 8:06:33 PM [9.827s]
                           
16:06:33 07:52           [update-upstream-analysis]
16:06:38 07:57           [trim-downstream-analysis]
16:06:43 08:02         [zinc-finalize]
16:06:43 08:02         [apt-post]
16:06:43 08:02         [java-post]
16:06:43 08:02         [zinc-post]
16:06:43 08:02     [jvm-dep-check]
16:06:44 08:03       [cache]   
                   No cached artifacts for 3 targets.
                   Invalidated 3 targets.
                   Caching artifacts for 3 targets.
  • b

    bored-art-40741

    09/08/2015, 8:16 PM
    It's also consuming an ungodly amount of RAM
  • b

    bored-art-40741

    09/08/2015, 8:16 PM
    I wonder if we're populating some intermediate data structure inefficiently
  • w

    witty-crayon-22786

    09/08/2015, 8:19 PM
    @bored-art-40741: i don't know what i'm looking at with that paste
  • b

    bored-art-40741

    09/08/2015, 8:19 PM
    The timestamps
  • w

    witty-crayon-22786

    09/08/2015, 8:19 PM
    and i also don't know whether we're talking about isolated or global
  • b

    bored-art-40741

    09/08/2015, 8:20 PM
    Global
  • w

    witty-crayon-22786

    09/08/2015, 8:20 PM
    hm
  • w

    witty-crayon-22786

    09/08/2015, 8:21 PM
    it's possible that it's all in product generation for the dep check?
  • w

    witty-crayon-22786

    09/08/2015, 8:22 PM
    ...hm, yea.
  • w

    witty-crayon-22786

    09/08/2015, 8:22 PM
    because it will regenerate the products for all targets
  • w

    witty-crayon-22786

    09/08/2015, 8:23 PM
    oh... i think the product 
    is_required
    predicate would help with this.
  • b

    bored-art-40741

    09/08/2015, 8:23 PM
    ah that makes sense
  • w

    witty-crayon-22786

    09/08/2015, 8:23 PM
    ...but maybe not. because the jvm dep check can't know which targets it would need to generate the products for until execute =(
  • b

    bored-art-40741

    09/08/2015, 8:23 PM
    Though! Still worth noting that this is not performant enough on a large graph
  • b

    bored-art-40741

    09/08/2015, 8:24 PM
    These guys are seeing 12+GB resident in python
  • b

    bored-art-40741

    09/08/2015, 8:24 PM
    It is building an ordered set of doom
  • w

    witty-crayon-22786

    09/08/2015, 8:24 PM
    the dep check? yea.
  • b

    bored-art-40741

    09/08/2015, 8:24 PM
    I've also got some progress on this isolated OOM
  • b

    bored-art-40741

    09/08/2015, 8:24 PM
    -.txt
    -.txt
  • b

    bored-art-40741

    09/08/2015, 8:25 PM
    That's a "possible deadlock found" warning from YourKit
  • b

    bored-art-40741

    09/08/2015, 8:25 PM
    That is not a deadlock, but it is a thread hung on classloading
  • b

    bored-art-40741

    09/08/2015, 8:25 PM
    I bumped the heap available from 6gb to 10gb
  • b

    bored-art-40741

    09/08/2015, 8:25 PM
    It is now happily eating 10gb and once again frozen attempting to classload and about to OOM
1...314315316...2420Latest