So far we haven't actually seen perf issues due to build graph operations specifically, only due to greedy BUILD file operations (globs and especially rglobs) and to some extent parsing the BUILD files