https://pantsbuild.org/ logo
h

hundreds-father-404

02/12/2021, 3:07 AM
@witty-crayon-22786 I get core dumps deterministically when trying to test remote execution locally in my devbox. are you still hoping for those?
w

witty-crayon-22786

02/12/2021, 6:03 PM
the backtrace is actually an interesting one
👀 1
Copy code
#0  0x00007fb9dadb7588 in __rust_probestack () from /data/home/eric/pants/src/python/pants/engine/internals/native_engine.so
#1  0x00007fb9d9eb4b8a in task_executor::Executor::future_with_correct_context (future=<error reading variable: value of type `core::future::from_generator::GenFuture<generator-0>' requires 120728 bytes, which is more than max-value-size>)
    at /data/home/eric/pants/src/rust/engine/task_executor/src/lib.rs:158
#2  0x00007fb9d9eb501e in task_executor::Executor::spawn (self=0x55a8d29a69b8, future=...) at /data/home/eric/pants/src/rust/engine/task_executor/src/lib.rs:104
#3  0x00007fb9d9f847dd in <process_execution::remote_cache::CommandRunner as process_execution::CommandRunner>::run::__run::{{closure}} () at process_execution/src/remote_cache.rs:509
...
👀 1
h

hundreds-father-404

02/12/2021, 6:04 PM
Is it insightful beyond my workunits change?
w

witty-crayon-22786

02/12/2021, 6:04 PM
maybe not. it’s panicing in the call to spawn in
future_with_correct_context
👍 1
…oh. which is … maybe a stack overflow?
h

hundreds-father-404

02/12/2021, 6:07 PM
iiuc, the Future is too large? We do have several layers of nested async futures being passed around
w

witty-crayon-22786

02/12/2021, 6:08 PM
yea. this is still not a solved problem. blugh.
i’m not sure if this calls for a convention (Box whenever you X), an investigation (is there a main culprit), etc
👍 1
f

fast-nail-55400

02/12/2021, 7:24 PM
last time I had to deal with such a thing, I just put in log statements until I narrowed down to which future was actually blowing up the stack and put a box there
w

witty-crayon-22786

02/12/2021, 7:26 PM
…i wonder if there is a way to “sizeof” them …?
f

fast-nail-55400

02/12/2021, 8:45 PM
I had tried to do that as well, but didn’t get anywhere with it. mainly because it doesn’t tell you what else is on the stack at the same time that together blow the stack up
it would be nice if Rust had an annotation similar to the type name limits to force an error if a generator/future goes over a certain size
👍 1
w

witty-crayon-22786

02/12/2021, 8:46 PM
i was thinking that you could pepper size calls at various boundaries, and then add/remove boxes until the sizes were reasonable
yea, this is awkward. two or three rust warts this week
h

hundreds-father-404

02/13/2021, 12:41 AM
Woot that fixed it! Thanks!
👍 1
w

witty-crayon-22786

03/19/2021, 3:14 AM
related: https://rust-lang.github.io/wg-async-foundations/vision/status_quo/alan_runs_into_stack_trouble.html … and there are actually already some tips in there.
Copy code
RUSTFLAGS = "-Zprint-type-sizes"