https://pantsbuild.org/ logo
Powered by
#general
  • b

    bored-art-40741

    01/26/2016, 7:25 PM
    ~/.cache/pants
    does not exist in jenkins
  • b

    bored-art-40741

    01/26/2016, 7:25 PM
    It's set to be within the executor directory
  • b

    bored-art-40741

    01/26/2016, 7:25 PM
    i.e. 
    $WORKSPACE/.cache/pants
  • b

    bored-art-40741

    01/26/2016, 7:26 PM
    My recent change moves the ivy resolution within .pants.d
  • a

    ancient-stone-50795

    01/26/2016, 7:26 PM
    Our CI build just used a separate os container for each shard, we never have problems like this.
  • b

    bored-art-40741

    01/26/2016, 7:26 PM
    Yes, but not everyone's is, and we don't claim to our users that they need to isolate CIs that aggressively
  • b

    bored-art-40741

    01/26/2016, 7:26 PM
    Therefore we shouldn't ignore the errors that not-isolating causes
  • a

    ancient-stone-50795

    01/26/2016, 7:26 PM
    We don't say much about CI at all.
  • b

    bored-art-40741

    01/26/2016, 7:27 PM
    What I'm testing right now seems to have fixed all of the ivy race issues, except maybe this one. But this one, maybe it's actually a resolution timeout or something?
  • a

    ancient-stone-50795

    01/26/2016, 7:27 PM
    FWIW, As far as AWS pricing goes, I thought it was just about as cheap to get lots of small containers as to run a single big one.
  • b

    bored-art-40741

    01/26/2016, 7:27 PM
    I'm suspicious that this is the one failing when all of the others were fixed by my change
  • r

    rough-minister-58256

    01/26/2016, 7:27 PM
    I’m =patrick - we should fix up this case because its likely others will encounter it in the future.
  • r

    rough-minister-58256

    01/26/2016, 7:27 PM
    however in terms of how we configure CI, we should probably still try our best to fully isolate those dirs
  • r

    rough-minister-58256

    01/26/2016, 7:28 PM
    primarily because it will prevent parallelism to share and lock around a shared ivy dir
  • b

    bored-art-40741

    01/26/2016, 7:28 PM
    I really want to ship pants in a way that the default way of using it out of the box isn't totally broken and racey
  • r

    rough-minister-58256

    01/26/2016, 7:28 PM
    fully behind that
  • b

    bored-art-40741

    01/26/2016, 7:28 PM
    I think it's possible, and I don't think this should be a controversial priority
  • b

    bored-art-40741

    01/26/2016, 7:29 PM
    I think I've got all of the ivy parallelism stuff fixed with this change
  • b

    bored-art-40741

    01/26/2016, 7:29 PM
    It uses ivy's native locking and moves the resolution scratch dir out from being global. Only the artifacts are global, and they're supposed be concurrency safe
  • b

    bored-art-40741

    01/26/2016, 7:29 PM
    Which makes me suspect that one of two things is happening with this failure
  • b

    bored-art-40741

    01/26/2016, 7:29 PM
    (1) We're running ivy in parallel within one process or nailgun, violating our own local ivy resolution concurrency promise
  • b

    bored-art-40741

    01/26/2016, 7:30 PM
    (And I could further isolate this probably by just always using a tmpdir or something)
  • b

    bored-art-40741

    01/26/2016, 7:30 PM
    (2) This is a timeout that just happens sometimes
  • b

    bored-art-40741

    01/26/2016, 7:31 PM
    Uh oh. I think my spot instances might have been preempted, interesting
  • b

    bored-art-40741

    01/26/2016, 7:31 PM
    Never happened before
  • j

    jolly-chef-92794

    01/26/2016, 7:37 PM
    @ancient-stone-50795, @witty-crayon-22786, @bored-art-40741, opinions on how merging 
    managed_jar_dependencies
    should work? (See @happy-kitchen-89482’s last comment on https://rbcommons.com/s/twitter/r/3367/)
  • a

    ancient-stone-50795

    01/26/2016, 7:44 PM
    The amount of code needed to add this feature back in if needed is small, so why don't we just simplify the PR and remove that option for now, and then we can add it back and make a case for it if we have a concrete reason to use it in our repo.
  • b

    bored-art-40741

    01/26/2016, 7:48 PM
    Sigh, worker preemption seems like it's going to be a real thing
  • b

    bored-art-40741

    01/26/2016, 7:48 PM
    Maybe I should rethink using spot instances
  • b

    bored-art-40741

    01/26/2016, 7:49 PM
    @jolly-chef-92794: I've not really had the time to keep up with the specifics of this. I think it's good that we're trying to formalize pinning in a sane way, but the issues that Benjy has raised are valid
1...914915916...2420Latest