https://pantsbuild.org/ logo
Powered by
#general
  • f

    freezing-motorcycle-14222

    08/31/2015, 10:59 PM
    Yeah, 
    ./pants run
    brought the time down to about 4-5 seconds
  • e

    enough-analyst-54434

    08/31/2015, 10:59 PM
    and likewise on run 2?
  • e

    enough-analyst-54434

    08/31/2015, 11:00 PM
    that's way higher than I'd expect
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:00 PM
    Yes
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:00 PM
    Is there a way I can collect timings of the different steps? I'm noticing it stall on a couple of steps by eye
  • e

    enough-analyst-54434

    08/31/2015, 11:00 PM
    -x
  • e

    enough-analyst-54434

    08/31/2015, 11:00 PM
    or ./pants server
  • e

    enough-analyst-54434

    08/31/2015, 11:00 PM
    but the logging for the python stuff may not be fine grained enout
  • e

    enough-analyst-54434

    08/31/2015, 11:01 PM
    another easy thing to add though
    Copy code
    with self.context.new_workunit...
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:01 PM
    Copy code
    2.287 background
1.629 compute-shared-props
0.689 main:compile:library-buildprops
0.494 main:run:py:run
0.338 main
0.141 main:compile
0.140 main:compile:validate_java_version
  • e

    enough-analyst-54434

    08/31/2015, 11:01 PM
    grep should turn up lots of those in the codebase for examples
  • e

    enough-analyst-54434

    08/31/2015, 11:01 PM
    the background represents - for the most part - shell-outs not tracked in a workunit
  • e

    enough-analyst-54434

    08/31/2015, 11:02 PM
    what is that top one under background?
  • e

    enough-analyst-54434

    08/31/2015, 11:02 PM
    that's Twitter custom IIUC
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:02 PM
    I'll check under pants_internal, then
  • r

    rough-minister-58256

    08/31/2015, 11:03 PM
    Copy code
    class ComputeProps(Task):
  """A task that fires off background work to compute various git properties.
  • e

    enough-analyst-54434

    08/31/2015, 11:03 PM
    Its hard to imagine how your trick gets around that?
  • e

    enough-analyst-54434

    08/31/2015, 11:03 PM
    how does that not bump your 0.5 up > 2s?
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:03 PM
    We're basically skipping pants by caching the generated PEX
  • e

    enough-analyst-54434

    08/31/2015, 11:03 PM
    aha - so a complete hack
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:03 PM
    Yeah simple smile
  • e

    enough-analyst-54434

    08/31/2015, 11:03 PM
    gotcha
  • e

    enough-analyst-54434

    08/31/2015, 11:04 PM
    I just hope you can impart your efforts to make things actually better for all to your surrounding ragers
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:04 PM
    We basically have a directory that is a map from MD5 to PEX
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:04 PM
    Yeah, that's the whole reason I was asking, to see if I could improve things upstream
  • e

    enough-analyst-54434

    08/31/2015, 11:05 PM
    So, I'd hit binary used cached_chroots, and then optimize the ComputeProps thing which - for a pydc of "background" takes up a substantial foreground time
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:05 PM
    Alright
  • r

    rough-minister-58256

    08/31/2015, 11:05 PM
    is your git in good shape? source-optimized etc?
  • f

    freezing-motorcycle-14222

    08/31/2015, 11:06 PM
    Possibly not. I remember Jonathan Simms talking about that, but I thought it was rolled out transparently to the whole company by now
  • r

    rough-minister-58256

    08/31/2015, 11:07 PM
    think its manual right now - but git operations via pants are presumably gated by git performance
1...260261262...2420Latest