hey we just bumped our pants version to 2.1.1rc1 to test out the fix for https://github.com/pantsbuild/pants/issues/11201 but now running a pants test on the target that was hitting the maxrecursion limit now results in pants hanging/spinning. The last few trace logs are like the following:

12:59:32.16 [TRACE] Completed: Inferring Python dependencies.
12:59:32.16 [TRACE] Completed: Inferring dependencies on `__init__.py` files
12:59:32.16 [TRACE] Completed: Inferring dependencies on `__init__.py` files
12:59:32.16 [TRACE] Completed: Resolve direct dependencies
12:59:32.16 [TRACE] Completed: Resolve direct dependencies
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_unexpanded_targets
12:59:32.16 [TRACE] Completed: Inferring Python dependencies.
12:59:32.16 [TRACE] Completed: Inferring dependencies on `__init__.py` files
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_unexpanded_targets
12:59:32.16 [TRACE] Completed: Inferring Python dependencies.
12:59:32.16 [TRACE] Completed: Inferring dependencies on `__init__.py` files
12:59:32.16 [TRACE] Completed: Resolve direct dependencies
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_targets
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_targets
12:59:32.16 [TRACE] Completed: Resolve direct dependencies
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_unexpanded_targets
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_unexpanded_targets
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_targets
12:59:32.16 [TRACE] Completed: pants.engine.internals.graph.resolve_targets

I’m running

pants -ltrace --no-pantsd test …

if there’s additional debug flags i can set to get more insight into what it’s doing let me know

cc @witty-crayon-22786

argh. sorry folks.

How long did you wait before ctrl-cing? I’m curious if it does eventually evaluate. Of course, that’s not desirable to take that long to evaluate and this needs to be revisited again, but may help to understand what’s going on.

i have left it for probably over 10 minutes before hitting ctrl-c

that’s definitely my fault. should have reproduced the bug before attempting to fix it.

and yah the cpu gets pegged when it hits this so smells of some infinite loop

so. regarding reproducing the original issue: i’ll need a bit more information

What’s surprising to me is that it does evaluate in Python - doesn’t hang if you tweak the recursion limit. Iiuc, we didn’t tweak the algorithm and only ported it to Rust

yah i saw the commit, seemed straightforward enough

the thing that is glaringly missing here is an easier way to dump your entire graph of dependencies without running

./pants dependencies

once per file.

i could dump the deps with --transitive for the test target no?

yes, but that wouldn’t get us the intermediate graph shape… the direct dependencies involved, basically.

@witty-crayon-22786 the closet we have to that is

./pants dependees --transitive --output-format=json ::

, which shows per-file. Would that help? You’d need to run on 2.1.0, as the

--transitive

flag is the part triggering the cycle

oh! awesome. yes, that’s exactly what we need.

ok, will do

@helpful-lunch-92084: if you feel comfortable sharing a (sanitized) version of that output, and identifying the problematic target, that would be very helpful.

yah let me double check on that

got it. yea, the json formatted output is the thing we really need: https://pantsbuild.slack.com/archives/C046T6T9U/p1606760315040400?thread_ts=1606759358.037000&cid=C046T6T9U

Interesting, huh. Was the failing command

./pants test $bad_target

? I would expect that to behave the same as

./pants dependencies --transitive $bad_target

. It should be using the exact same problematic code path

if it’s sensitive, you can DM it and i can sanitize it.

ok, just curious why i would need to do it on 2.1.0?

That command might work on 2.1.1rc1, and that’s fine if it does! I only anticipated you would get the same hang on rc1 because I suspect the code path to resolve transitive deps is what’s broken

oh i see it was dependees not dependencies

Yeah, but

dependees --transitive

uses the same code path as well for transitive deps as used by

test

and` dependencies --transitive`. So it’s fishy if it does not hang on rc1, but that’s useful data what happens.

yah dependees works on 2.1.1rc1 as well

thanks a ton, and sorry for the trouble.

it’s ok, we can revert to 2.1.0 in the meantime