https://pantsbuild.org/ logo
Powered by
#general
  • b

    bored-art-40741

    10/05/2015, 4:54 PM
    And you couldn't know that C was cached because you rely on the output of A and B instead of relying on A and B to have the same output for the same inputs
  • w

    witty-crayon-22786

    10/05/2015, 4:55 PM
    i do like the idea of being able to hit the cache top down. but i don't know of any real world examples where that is possible
  • b

    bored-art-40741

    10/05/2015, 4:56 PM
    It's possible basically 100% of the time except in corner cases like ivy
  • b

    bored-art-40741

    10/05/2015, 4:56 PM
    Where global knowledge is being implicitly used even though the build only operates over a subgraph
  • w

    witty-crayon-22786

    10/05/2015, 4:56 PM
    mm, yea. in the sense that for a bundle you can skip straight to fetching the bundle.
  • b

    bored-art-40741

    10/05/2015, 4:57 PM
    Yeah
  • w

    witty-crayon-22786

    10/05/2015, 4:57 PM
    hm
  • w

    witty-crayon-22786

    10/05/2015, 4:57 PM
    so actually
  • b

    bored-art-40741

    10/05/2015, 4:57 PM
    It's important to note that the "fingerprint" of a target here is still annotated by the type/name of the product in addition to the tasks that produced it
  • w

    witty-crayon-22786

    10/05/2015, 4:57 PM
    @fancy-queen-20734: what if rather than looking at the "resolved version" (ie, output)
  • b

    bored-art-40741

    10/05/2015, 4:57 PM
    So as long as we require determinism, we should always be able to compute fingerprints upfront
  • w

    witty-crayon-22786

    10/05/2015, 4:58 PM
    @fancy-queen-20734: we just looked at the inputs?
  • w

    witty-crayon-22786

    10/05/2015, 4:58 PM
    would preserve this case
  • w

    witty-crayon-22786

    10/05/2015, 4:58 PM
    @bored-art-40741: but we still need task fingerprint strategies to be transitive
  • w

    witty-crayon-22786

    10/05/2015, 4:58 PM
    somehow
  • b

    bored-art-40741

    10/05/2015, 4:59 PM
    I'm not sure what you mean
  • b

    bored-art-40741

    10/05/2015, 4:59 PM
    They already are
  • w

    witty-crayon-22786

    10/05/2015, 4:59 PM
    no, they're not
  • w

    witty-crayon-22786

    10/05/2015, 4:59 PM
    if ivy resolve comes up with one cache key, it has zero effect on jvm_compile
  • w

    witty-crayon-22786

    10/05/2015, 4:59 PM
    @bored-art-40741: sorry... transitive in the task graph
  • b

    bored-art-40741

    10/05/2015, 4:59 PM
    Ah
  • w

    witty-crayon-22786

    10/05/2015, 5:00 PM
    that's the real bug here... the fact that we fingerprint products was a distraction.
  • w

    witty-crayon-22786

    10/05/2015, 5:00 PM
    (fixable, it seems)
  • b

    bored-art-40741

    10/05/2015, 5:01 PM
    So, I strongly believe that the root issue here is that ivy resolution operates over a subgraph and then produces a single output. It happens that we then split up that output and assign bits and pieces of it to bits and pieces of the subgraph. However, from a caching perspective, we should really only cache the result for the entire subgraph and not individual nodes, or super graphs
  • b

    bored-art-40741

    10/05/2015, 5:02 PM
    I understand this has performance implications
  • b

    bored-art-40741

    10/05/2015, 5:02 PM
    And we might need to find a way around it
  • b

    bored-art-40741

    10/05/2015, 5:02 PM
    But I think that's the point in our ideal model where we're running into issues
  • w

    witty-crayon-22786

    10/05/2015, 5:02 PM
    @bored-art-40741: hm, not sure.
  • w

    witty-crayon-22786

    10/05/2015, 5:02 PM
    @bored-art-40741: imagine that rather than ivy resolve
  • w

    witty-crayon-22786

    10/05/2015, 5:03 PM
    @bored-art-40741: this was a huge binary
1...496497498...2420Latest