Unfortunately this behavior is a fundamental property of ivy changing its resolution based on (potentially small) changes in the subgraph that it's being resolved against