heyas, I'm working on automatically retrying tests...
# development
c
heyas, I'm working on automatically retrying tests, and I'm having trouble understanding how Pants doesn't cache failed test results. seems
class TestResult(EngineAwareReturnType)
has
cacheable
set to False. but naively retrying the
Get(TestResult, ...)
doesn't result in the re-execution of the rules in that path.
ah,
Process.cache_scope
on the process that runs pytest. Set to
ProcessCacheScope.PER_SESSION
, so trying to run it again won't work.
weird that changing it to
ProcessCacheScope.SUCCESSFUL
results in it still being cached, even on failure...
wait is that option only used for remote caching?
Copy code
fn cacheable(&self) -> bool {
    match self {
      NodeKey::Task(s) => s.task.cacheable,
      &NodeKey::SessionValues(_) | &NodeKey::RunId(_) => false,
      _ => true,
    }
  }
I guess since I'm executing a process I have an ExecuteProcess so it's always cached in the run-graph?
b
IIRC the results cacheable is about rendering. The process object determines if it's cached on failure (which shouldn't be default, as that should be rare)
And PER_SESSION means essentially (every run of Pants reruns), and that value is only used if the force flag is given (which makes sense since we want it to run then)
Changing it to SUCCESSFUL most certainly shouldn't cache it
Oh but I think you're on to something. I think everything is cached at the session level 🥴
c
yeah. I saw something about output_cacheable, but didn't have the chance to look into whether that was calculated after the output was known. I also don't know if there's a way to dirty that node so it runs again
b
No, that's a Stu question, I think. My naive brain says you'd need another field in a Process.
Or another value for the ProcessCacheScope
New field makes sense to me, as presumably more than tests could warrant a retry (like a process that talks to the network)