https://pantsbuild.org/ logo
Powered by
#general
  • w

    witty-crayon-22786

    05/04/2016, 8:19 PM
    so "tasks won’t have to manually load products", because they're already available in the graph
  • w

    witty-crayon-22786

    05/04/2016, 8:19 PM
    but they're not all serialized currently, because we don't always spill to disk. it remains to be seen what we do there in the case of things that we don't want to "cache" per-say, but also don't want to hold in memory.
  • h

    happy-kitchen-89482

    05/04/2016, 9:18 PM
    I’m thinking of the example of, say, the classes_by_source product
  • h

    happy-kitchen-89482

    05/04/2016, 9:18 PM
    That gets produced when you compile
  • h

    happy-kitchen-89482

    05/04/2016, 9:18 PM
    but zinc_compile.py needs to read them from disk even for the targets it’s not compiling because they haven’t changed
  • h

    happy-kitchen-89482

    05/04/2016, 9:18 PM
    How does that work in the brave new world?
  • h

    happy-kitchen-89482

    05/04/2016, 9:19 PM
    See JvmCompile.compute_classes_by_source
  • w

    witty-crayon-22786

    05/04/2016, 9:41 PM
    @happy-kitchen-89482: currently that would end up in memory, yes.
  • h

    happy-kitchen-89482

    05/04/2016, 9:41 PM
    “end up” how?
  • w

    witty-crayon-22786

    05/04/2016, 9:41 PM
    it would be the output of a task, and all tasks outputs are data in the product graph
  • w

    witty-crayon-22786

    05/04/2016, 9:42 PM
    there will presumably be a zinc-analysis product in the graph
  • w

    witty-crayon-22786

    05/04/2016, 9:42 PM
    and the classes by source are something that can be computed from the zinc analysis
  • w

    witty-crayon-22786

    05/04/2016, 9:45 PM
    again, it's somewhat orthogonal, but one of the things that is not nailed down yet is what happens when a product in the graph needs to be on disk for some reason (processing by an external process, consumption by code that needs random access to a file, etc)
  • w

    witty-crayon-22786

    05/04/2016, 9:45 PM
    as it stands, everything is in memory.
  • w

    witty-crayon-22786

    05/04/2016, 9:47 PM
    and yes, this is a gap. but it's tightly related to what child process management looks like, and that's the next thing we're working on.
  • b

    bored-art-40741

    05/04/2016, 10:11 PM
    @rough-minister-58256: The loop perf test response is incorrect
  • r

    rough-minister-58256

    05/04/2016, 10:11 PM
    oh?
  • b

    bored-art-40741

    05/04/2016, 10:11 PM
    range
    starts with 
    0
  • b

    bored-art-40741

    05/04/2016, 10:11 PM
    all()
    immediately short circuits
  • r

    rough-minister-58256

    05/04/2016, 10:11 PM
    ah heh
  • r

    rough-minister-58256

    05/04/2016, 10:12 PM
    my bad
  • b

    bored-art-40741

    05/04/2016, 10:12 PM
    The for loop is in fact faster
  • b

    bored-art-40741

    05/04/2016, 10:12 PM
    3-4x
  • r

    rough-minister-58256

    05/04/2016, 10:12 PM
    welp!
  • b

    bored-art-40741

    05/04/2016, 10:13 PM
    If you are a truly looking for fastest (which I don't think is necessary), the trick is much nastier
  • b

    bored-art-40741

    05/04/2016, 10:13 PM
    You make a dequeue with a max length of 0 or 1 (forget which) and then use its init
  • b

    bored-art-40741

    05/04/2016, 10:13 PM
    It has a special C function for generator unrolling
  • r

    rough-minister-58256

    05/04/2016, 10:14 PM
    will amend the RB - nice catch
  • b

    bored-art-40741

    05/04/2016, 10:14 PM
    Oh you know, not sure if for is actually faster
  • b

    bored-art-40741

    05/04/2016, 10:14 PM
    Worth noting your perf test, other than the short circuit, gimped the 
    all()
    solution by putting it within another generator comp
1...124612471248...2420Latest