https://pantsbuild.org/ logo
Powered by
#general
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:33 PM
    Option registration can still force the argument parsers to be evaluated even when behind a lazy val
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:33 PM
    I don't know if there's logic in pants to make argument registration lazy on a per-command basis
  • e

    enough-analyst-54434

    09/24/2015, 10:34 PM
    s/1st call/before 1st call/ - there should be 1 large overhead to gathering all the aliases
  • h

    happy-kitchen-89482

    09/24/2015, 10:34 PM
    Yes
  • h

    happy-kitchen-89482

    09/24/2015, 10:34 PM
    that's exactly what I'm seeing
  • h

    happy-kitchen-89482

    09/24/2015, 10:34 PM
    create_call_proxy gets called hundreds of times
  • h

    happy-kitchen-89482

    09/24/2015, 10:34 PM
    and it allocates a lot of stuff
  • h

    happy-kitchen-89482

    09/24/2015, 10:35 PM
    35% of that second is in abc.new
  • h

    happy-kitchen-89482

    09/24/2015, 10:35 PM
    amazingly
  • e

    enough-analyst-54434

    09/24/2015, 10:35 PM
    that makes sense, that just says are backends are too big and/or we should use source_roots to filter the call proxies we get ready
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:35 PM
    Here are the big ones that I found:
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:35 PM
    How do I paste code in Slack?
  • e

    enough-analyst-54434

    09/24/2015, 10:36 PM
    put it in a github issue
  • e

    enough-analyst-54434

    09/24/2015, 10:36 PM
    If we're going to start to collect perf tweaks it would be awesome to have the data in a useable place
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:36 PM
    Alright, I'll open something up with the data I collected
  • e

    enough-analyst-54434

    09/24/2015, 10:36 PM
    That would be great - ty
  • h

    happy-kitchen-89482

    09/24/2015, 10:37 PM
    Now I'm looking at a profile of 
    ./pants list
    run in Foursquare's repo (takes 30 seconds or more).
  • h

    happy-kitchen-89482

    09/24/2015, 10:37 PM
    initialize_parse_state is still ~19% of time
  • h

    happy-kitchen-89482

    09/24/2015, 10:37 PM
    But other aspects of BUILD file parsing are now more dominant
  • h

    happy-kitchen-89482

    09/24/2015, 10:38 PM
    40% of time is spent inside os.walk/os.isdir etc.
  • h

    happy-kitchen-89482

    09/24/2015, 10:38 PM
    lots of stat-ing I guess.
  • h

    happy-kitchen-89482

    09/24/2015, 10:40 PM
    One call to scan_buildfiles is generating 200K recursive to os.walk. What? Maybe it's walking into .git or something.
  • e

    enough-analyst-54434

    09/24/2015, 10:41 PM
    So 
    addressable_factory = TargetAddressable.factory(target_type=tgt_type, alias=tgt_alias)
    could be lifted out of 
    initialize_parse_state
    and memoized over all BUILD parses
  • e

    enough-analyst-54434

    09/24/2015, 10:41 PM
    that would save some allocations
  • f

    freezing-motorcycle-14222

    09/24/2015, 10:42 PM
  • e

    enough-analyst-54434

    09/24/2015, 10:42 PM
    And 
    parse_context = ParseContext(rel_path=build_file.spec_path, type_aliases=type_aliases)
    <- the type_aliases there could be lifted as well
  • e

    enough-analyst-54434

    09/24/2015, 10:43 PM
    Those are the only 2 spots though fwict
  • e

    enough-analyst-54434

    09/24/2015, 10:43 PM
    FWIW @freezing-motorcycle-14222 - tick fences like github - triple `
  • e

    enough-analyst-54434

    09/24/2015, 10:43 PM
    ```
  • h

    happy-kitchen-89482

    09/24/2015, 10:43 PM
    I don't know the build file parsing code well enough yet, but will poke around.
1...458459460...2420Latest