https://pantsbuild.org/ logo
Powered by
#general
  • b

    bored-art-40741

    01/26/2016, 1:47 AM
    Right, making it non-concurrent is easy
  • b

    bored-art-40741

    01/26/2016, 1:47 AM
    The problem is that each worker has ~5 executors
  • b

    bored-art-40741

    01/26/2016, 1:48 AM
    And they kick off all of their shards in parallel, otherwise the worker wouldn't have been brought up
  • b

    bored-art-40741

    01/26/2016, 1:48 AM
    Which means a machine suddenly starts hammering Maven/Nexi for artifacts
  • b

    bored-art-40741

    01/26/2016, 1:48 AM
    Their load balancers tend to frown on that use pattern
  • b

    bored-art-40741

    01/26/2016, 1:49 AM
    I wish I could make "most" of the blobs themselves into an EBS volume so 95% of the downloading is mitigated
  • b

    bored-art-40741

    01/26/2016, 1:49 AM
    Or have everything go through a local varnish or something
  • b

    bored-art-40741

    01/26/2016, 1:49 AM
    But we don't really have a good mechanism in place for that
  • b

    bored-art-40741

    01/26/2016, 1:49 AM
    Hence the issue
  • b

    bored-art-40741

    01/26/2016, 1:52 AM
    Of course the fundamental issue here is that we're using a workspace meant to store blobs also as a concurrently-accessible scratch space for storing resolution metadata
  • b

    bored-art-40741

    01/26/2016, 1:53 AM
    And naturally Ivy has an option to allow you to lock around this behavior
  • b

    bored-art-40741

    01/26/2016, 1:53 AM
    ...that doesn't work!
  • b

    bored-art-40741

    01/26/2016, 1:58 AM
    Ha, I owe John a coke
  • b

    bored-art-40741

    01/26/2016, 2:04 AM
    @enough-analyst-54434: Are you around to slack for 5 minutes?
  • b

    bored-art-40741

    01/26/2016, 3:25 AM
    Monkeying with jenkins right now, don't be alarmed if you trigger a CI and it breaks!
  • b

    bored-art-40741

    01/26/2016, 3:32 AM
    Another easy but annoying bug we need to fix: https://github.com/pantsbuild/pants/issues/2719
  • b

    bored-art-40741

    01/26/2016, 3:33 AM
    Added starter and bug tags
  • b

    bored-art-40741

    01/26/2016, 3:45 AM
  • b

    bored-art-40741

    01/26/2016, 3:45 AM
    Apparently we don't have pex set up to respect our bootstrap dir
  • b

    bored-art-40741

    01/26/2016, 3:45 AM
    Copy code
    export PANTS_BOOTSTRAPDIR=/jenkins/workspace/pants_ci.contrib/.cache/pants
  • b

    bored-art-40741

    01/26/2016, 3:46 AM
    (That's per-workspace, so there's no concurrency issue within the workspace itself)
  • b

    bored-art-40741

    01/26/2016, 3:46 AM
    But parallel pex "resolves" are also a landmine that easily generates racey failures
  • b

    bored-art-40741

    01/26/2016, 3:48 AM
    Ah, and failures due to hard coded expected ivy outputs: http://jenkins.pantsbuild.org/job/pants_ci.integration_tests/SHARD=1/23/consoleFull
  • b

    bored-art-40741

    01/26/2016, 3:50 AM
  • b

    bored-art-40741

    01/26/2016, 3:51 AM
    Seems like maybe IvyUtils isn't aware of changes to ivysettings.xml
  • l

    limited-country-20626

    01/26/2016, 5:17 AM
    From the peanut gallery: At this point I like RB the most of any review tool, although probably not enough to support any significant cost. And I think being an Apache project would be great.
  • b

    bored-art-40741

    01/26/2016, 5:48 AM
    Okay, I'll have to chug more on the integration test pants.ini config later. I got pretty far, but integration tests still ignore "default" pants.ini config
  • b

    bored-art-40741

    01/26/2016, 5:48 AM
    See https://github.com/pantsbuild/pants/pull/2812 for what I've done so far to fiddle with it
  • b

    bored-art-40741

    01/26/2016, 5:50 AM
    Basically all integration tests fail, because they expect one of the ivy caches to be in a directory that doesn't reach their config via pants.ini
  • b

    bored-art-40741

    01/26/2016, 5:50 AM
1...910911912...2420Latest