https://pantsbuild.org/ logo
Powered by
#development
  • a

    aloof-angle-91616

    05/04/2020, 11:40 PM
    if ulimit doesn't fix it, i was thinking that would rule out actual fd count
  • a

    aloof-angle-91616

    05/04/2020, 11:40 PM
    if ulimit does fix it, then i would be pretty sure that that particular instance was a result of actual fd count
  • h

    hundreds-father-404

    05/04/2020, 11:41 PM
    Also, the weird thing is, this morning resetting 
    ulimit -n
    fixed it. I have every reason to believe it was already 20k because of my .zshrc, so i don’t know why this fixed it. I didn’t change the value - just reset it. But, @happy-kitchen-89482’s error kept happening even with a machine restart and resetting the value.
  • a

    aloof-angle-91616

    05/04/2020, 11:41 PM
    the machine restart part is weird
  • a

    aloof-angle-91616

    05/04/2020, 11:42 PM
    one possible scenario might be: (1) pants allocates lots of stuff in lmdb (2) machine restart (3) pants maps the same stuff into lmdb immediately when it starts up again after restart
  • a

    aloof-angle-91616

    05/04/2020, 11:43 PM
    there should be documentation about what to do when hitting resource limits, but possibly not in the 
    lmdb
    crate itself, rather for the lmdb c library
  • h

    happy-kitchen-89482

    05/05/2020, 12:59 AM
    FWIW I'm seeing this running in the pants repo (so no pantsd)
  • h

    happy-kitchen-89482

    05/05/2020, 12:59 AM
    E.g., by running 
    ./pants test contrib/go/tests/python/pants_test/contrib/go/tasks
  • h

    happy-kitchen-89482

    05/05/2020, 1:00 AM
    with that ulimit set to 10k
  • a

    aloof-angle-91616

    05/05/2020, 1:00 AM
    pantsd is what would help this, potentially, since it does some gc (of lmdb on disk, not in memory) in the background
  • h

    happy-kitchen-89482

    05/05/2020, 1:00 AM
    Note that the command was 
    ./pants
    , running v1 tasks...
    👍 2
    h
    w
    • 3
    • 10
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:10 PM
    has anyone attached a debugger to running patns before for the purpose of stepping through rust code?
  • h

    hundreds-father-404

    05/06/2020, 8:13 PM
    No, not Rust code. Only Python
  • w

    witty-crayon-22786

    05/06/2020, 8:36 PM
    iirc, Twitter had a page about loading pants in Intellij… maybe @wide-energy-11069 would be able to share that
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:38 PM
    I saw that link, but the specific thing I'm trying to do is hook up to a running pants instance and see where a specific delay is coming from
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:39 PM
    I actually found out that you can just run 
    gdp -p <pid>
    against the pants process, and that will get you to the normal gdb repl
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:39 PM
    but the function I want to see the name of in the backtrace has ??
  • w

    witty-crayon-22786

    05/06/2020, 8:40 PM
    since we have a few “optimize rust intrinsic X for Snapshots” items coming up right now (and with the caveat that this is “optimizing before having profiled”), i should mention that way back in the day, @average-vr-56795 and i discussed having an in-memory-optimized form of 
    DirectoryDigest
    / 
    Snapshot
    , which would effectively be a 
    HashMap<PathBuf, (File|Directory)>
    to make those kinds of functions easier to write. currently we lazily load from the database, but i think that that has made a lot of things harder to implement efficiently.
    h
    • 2
    • 24
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:40 PM
    and giving gdb the path to 
    rust/engine/target/debug/libengine_cffi.so
    doesn't seem to help
  • w

    witty-crayon-22786

    05/06/2020, 8:42 PM
    which page do you mean? i mean specifically for Rust
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:47 PM
    page?
  • w

    witty-crayon-22786

    05/06/2020, 8:47 PM
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:48 PM
  • h

    hundreds-breakfast-49010

    05/06/2020, 8:48 PM
    oh I guess that's not quite what you were talking about
  • w

    witty-crayon-22786

    05/06/2020, 9:07 PM
    right.
  • w

    witty-crayon-22786

    05/06/2020, 9:08 PM
    cc @early-needle-54791: i know you’re an emacs user, but do you know the page i’m talking about?
  • e

    early-needle-54791

    05/06/2020, 9:46 PM
    I use vim! 🙈 But I don’t recall a page about loading pants into IJ.
  • w

    witty-crayon-22786

    05/06/2020, 9:46 PM
    its on go/builddocs, iirc.
  • h

    hundreds-breakfast-49010

    05/06/2020, 9:46 PM
    I use vim as well, so I'd be unliekly to switch to IJ unless this was the only way to get this to work
  • h

    hundreds-breakfast-49010

    05/06/2020, 9:47 PM
    but actually I've had a surprising amount of success with 
    gdb -p
1...727728729...906Latest