https://pantsbuild.org/ logo
Powered by
#development
  • r

    rough-minister-58256

    03/20/2017, 10:57 PM
    then guessing peiyu should rebase his PR on top of that
  • w

    witty-crayon-22786

    03/20/2017, 10:57 PM
    indeed
  • f

    full-kilobyte-93429

    03/20/2017, 10:57 PM
    yea, rebasing
  • r

    rough-minister-58256

    03/20/2017, 10:57 PM
    and will give it another go
  • w

    witty-crayon-22786

    03/20/2017, 10:58 PM
    @full-kilobyte-93429 : er, another thing landed a second ago, sorry.
  • r

    rough-minister-58256

    03/20/2017, 10:59 PM
    whoa ya
    👍 1
  • r

    rough-minister-58256

    03/20/2017, 10:59 PM
    confirmed:
  • r

    rough-minister-58256

    03/20/2017, 10:59 PM
    Copy code
    RSS=3.4GB
real    8m9.703s
user    8m16.123s
sys     0m31.198s
  • r

    rough-minister-58256

    03/20/2017, 10:59 PM
    thats awesome
  • r

    rough-minister-58256

    03/20/2017, 10:59 PM
    for ff54411
  • w

    witty-crayon-22786

    03/20/2017, 11:00 PM
    nice.
  • r

    rough-minister-58256

    03/20/2017, 11:00 PM
    🚀
  • r

    rough-minister-58256

    03/20/2017, 11:05 PM
    @full-kilobyte-93429 lmk when you’re rebased on fd3f8a8 and I can rerun your PR again and see if things improve
  • f

    full-kilobyte-93429

    03/20/2017, 11:06 PM
    yea, in a sec, just resolved some conflict, testing
  • r

    rough-minister-58256

    03/20/2017, 11:06 PM
    will need to 
    ./pants
    to gen a new native_engine_version too
  • w

    witty-crayon-22786

    03/20/2017, 11:08 PM
    one thing that should also be possible after 
    ff54411
    is using 
    PANTS_PROFILE
    to capture stuff happening on the main thread (which should be nearly everything)
  • w

    witty-crayon-22786

    03/20/2017, 11:08 PM
    so one theory would be that pre/post the 
    SelectTransitive
    change, all of the time related to merging lists should be gone
  • w

    witty-crayon-22786

    03/20/2017, 11:09 PM
    i can understand the fact that 
    loop_fn
    removes concurrency though... it basically introduces synchronization points at every level in the graph
  • w

    witty-crayon-22786

    03/20/2017, 11:10 PM
    ie, expand everything at level 
    N
    before beginning to expand level 
    N+1
  • f

    full-kilobyte-93429

    03/20/2017, 11:10 PM
    @rough-minister-58256 https://github.com/pantsbuild/pants/pull/4349 updated
  • r

    rough-minister-58256

    03/20/2017, 11:10 PM
    👍
  • w

    witty-crayon-22786

    03/20/2017, 11:11 PM
    the annoying thing is that because concurrency, memoizing the recursion requires a mutex and Shared futures
  • w

    witty-crayon-22786

    03/20/2017, 11:11 PM
    which is basically what the 
    Graph
    is giving us currently
  • w

    witty-crayon-22786

    03/20/2017, 11:16 PM
    ...and moreover, because the memoization is custom in the OrderMap, portions of it aren't saved across runs. damn.
  • w

    witty-crayon-22786

    03/20/2017, 11:21 PM
    AND we lose cycle detection. blargh.
  • r

    rough-minister-58256

    03/20/2017, 11:21 PM
    😞
  • f

    full-kilobyte-93429

    03/20/2017, 11:25 PM
    so if possible we solve this we still have room improving file io too?
  • w

    witty-crayon-22786

    03/20/2017, 11:29 PM
    yea
  • w

    witty-crayon-22786

    03/20/2017, 11:29 PM
    but... it might just be that moving file io into the graph would be the right choice, heh.
  • w

    witty-crayon-22786

    03/20/2017, 11:30 PM
    i'll try memoizing it outside the graph.
1...285286287...906Latest