https://pantsbuild.org/ logo
Powered by
#general
  • q

    quiet-ice-72984

    03/02/2017, 9:59 PM
    We copy the ivy cache to, 
    ~/.ivy2/*
    all comes along, same exact location
  • w

    witty-crayon-22786

    03/02/2017, 9:59 PM
    hm
  • q

    quiet-ice-72984

    03/02/2017, 10:00 PM
    Its literally 
    /
    , just on another OS instance, but even same user name
  • w

    witty-crayon-22786

    03/02/2017, 10:01 PM
    ok. so rather than being about the cache, this is probably entirely about the content of 
    .pants.d
  • w

    witty-crayon-22786

    03/02/2017, 10:02 PM
    i don't really know anything about 
    .pants.d
    portability, so i don't know why anything would be invalidated
  • m

    mysterious-action-44812

    03/02/2017, 10:03 PM
    who here would know? if we could make it portable it’d go a long way for our deployments / release as then going from AMI to fully built binaries when there’s not much code change could be very fast, whereas right now it’s like 10 minutes
  • w

    witty-crayon-22786

    03/02/2017, 10:03 PM
    -ldebug
    would know
  • m

    mysterious-action-44812

    03/02/2017, 10:05 PM
    sure we can turn on debug and investigate more, I meant any of the devs around that are more familiar with the internals around portability / invalidation of 
    .pants.d
    ?
  • q

    quiet-ice-72984

    03/02/2017, 10:06 PM
  • q

    quiet-ice-72984

    03/02/2017, 10:06 PM
    I can do ^ again with 
    -ldebug
    , gimme a few mins
  • w

    witty-crayon-22786

    03/02/2017, 10:06 PM
    @mysterious-action-44812 : i can tell you that already: it's not intended to be portable.
  • w

    witty-crayon-22786

    03/02/2017, 10:06 PM
    BUT i do not have any idea why it would not work if the box was otherwise identical.
  • q

    quiet-ice-72984

    03/02/2017, 10:14 PM
    Are there any docs on exactly how the invalidation works? i.e. what hashes/files its looking for to know to no-op? Or will 
    -ldebug
    clearly show it?
  • q

    quiet-ice-72984

    03/02/2017, 10:16 PM
    I see this on the 3rd run, but not the 2nd
    Copy code
    DEBUG] Using previous resolve.
DEBUG] Parsing ivy report /build/.pants.d/ivy/f7f5752499af315ad05b6e146c08f74072799996/resolve-report-default.xml
  • w

    witty-crayon-22786

    03/02/2017, 10:16 PM
    it's looking at the transitive fingerprint of a target
  • w

    witty-crayon-22786

    03/02/2017, 10:16 PM
    but... yea, no docs on that.
  • q

    quiet-ice-72984

    03/02/2017, 10:17 PM
    Copy code
    22:12:45 00:10     [routes]
22:12:51 00:16   [jvm-platform-validate]
22:12:51 00:16     [jvm-platform-validate]
                   pants_backend_jvm_tasks_jvm_platform_analysis_JvmPlatformValidate will read from local artifact cache at /build/.cache/pants_backend_jvm_tasks_jvm_platform_analysis_JvmPlatformValidate
22:12:51 00:16       [cache]
                   No cached artifacts for 29 targets.
                   Invalidated 29 targets.
22:12:51 00:16   [resolve]
22:12:51 00:16     [ivy]
                   pants_backend_jvm_tasks_ivy_resolve_IvyResolve will read from local artifact cache at /build/.cache/pants_backend_jvm_tasks_ivy_resolve_IvyResolve
22:12:52 00:17       [cache]
                   No cached artifacts for 80 targets.
                   Invalidated 80 targets.
22:12:53 00:18       [ivy-resolve]
                   pants_backend_jvm_tasks_ivy_resolve_IvyResolve will write to local artifact cache at /build/.cache/pants_backend_jvm_tasks_ivy_resolve_IvyResolve
                   Caching artifacts for 80 targets.
  • q

    quiet-ice-72984

    03/02/2017, 10:17 PM
    2nd run ^
  • q

    quiet-ice-72984

    03/02/2017, 10:18 PM
    Hmm ok, so no docs on why that 
    jvm-platform-validate
    says 
    No cached artifacts for 29 targets
    ?
  • q

    quiet-ice-72984

    03/02/2017, 10:25 PM
    Should I try to find the person who owns this code? Seems like what we want, at least for figuring out why ivy doesnt noop: https://github.com/pantsbuild/pants/blob/5470b8650698945efa4fe7bc09f3fbc57ab8a149/src/python/pants/backend/jvm/tasks/ivy_task_mixin.py#L254
  • q

    quiet-ice-72984

    03/02/2017, 10:35 PM
    Here's the 2nd time resolve (i.e. 1st time on that box): https://gist.github.com/MasterDDT/34681f7f5d047849805d9a87a2bb4468
  • w

    witty-crayon-22786

    03/02/2017, 11:06 PM
    two things here.
  • w

    witty-crayon-22786

    03/02/2017, 11:07 PM
    1) jvm-platform-validate is not seeing a target fingerprint that matches what is in 
    .pants.d/build_invalidator
  • w

    witty-crayon-22786

    03/02/2017, 11:07 PM
    so it thinks that the target hasn't been built in this workspace
  • w

    witty-crayon-22786

    03/02/2017, 11:07 PM
    and then
  • w

    witty-crayon-22786

    03/02/2017, 11:07 PM
    after it sees that the target hasn't built in the workspace
  • w

    witty-crayon-22786

    03/02/2017, 11:08 PM
    2) it's checking the cache and not finding the artifact for a previous run
  • w

    witty-crayon-22786

    03/02/2017, 11:08 PM
    missing both 
    .pants.d
    and 
    .cache
    means the target fingerprint has probably changed.
  • q

    quiet-ice-72984

    03/02/2017, 11:22 PM
    So 
    build_invalidator
    seems like it just has tons of hashes
    Copy code
    root@ip-10-0-7-150:/build/.pants.d/build_invalidator/pants_backend_jvm_tasks_jvm_platform_analysis_JvmPlatformValidate/7 > cat thrift-lib.src.main.thrift.thrift-lib.hash
16b36b678058d3b20a06391deb41186c3a1ecd00-PlatformFingerprintStrategy.da39a3ee5e6b_53c4bb9c6553
  • q

    quiet-ice-72984

    03/02/2017, 11:23 PM
    I can verify, but im guessing restoring the disk image didnt change any of that
1...165716581659...2420Latest