Hi all, has anyone ever seen artificial and intermittent dependency cycles show up? e.g.

IntrinsicError: The dependency graph contained a cycle:
  @rule(pants.core.util_rules.source_files.determine_source_files()) <-
@rule(pants.engine.internals.graph.hydrate_sources(third_party/glog:glog))
@rule(pants.backend.shell.util_rules.shell_command.shell_command_in_sandbox())
@rule(pants.backend.shell.util_rules.shell_command.run_adhoc_result_from_target())
@rule(pants.core.util_rules.source_files.determine_source_files()) <-

As you can guess, we've thrown together C++ support with experiemental_shell_command rules. These aren't true dependency cycles, they only show up sometimes, more often for some than others. We suspect that the file-change detector gets a little confused. We have to get around them by using --no-local-cache, which is becoming less feasible as our repo grows. Any ideas?

does this happen on clean builds, or only after edit-then-rebuild?

only during editing

hm… sorry for the trouble. yea, it is probably a false cycle created while cleaning the nodes. fwiw:

--no-local-cache

is not actually what is solving the issue most likely: rather, restarting

pantsd

, which holds the graph in memory. changing a bootstrap option like that will recreate the scheduler. so faster next time would likely be to remove the

.pids

directory or

pkill pantsd

ah ok, thanks for the info. This is at 2.16.0rc1. I'm not quite sure how to give you a repro, and I'm not sure how related it is to our use of experimental_shell_command through macros (which produces files in e.g. sandbox dist/build that are consumed later). maybe not related, just what seems potentially different from other folk's stuff

yea, these are tricky. in theory we have handling for this case (and we haven’t had any reports in … a few years), so it’s hard to tell what it might be related to. how frequently does it occur?

for some with quick fingers or maybe slow disks (save file and run command), very frequently? I just saw it twice in 10 minutes, switching between branches

would you mind filing an issue about this with any relevant details that you can share? i think that it might already be fixed in the 2.17.x releases, but i’d want to stare at the code a bit to be sure, and the ticket would help with that

done, thanks for looking! https://github.com/pantsbuild/pants/issues/18992