the

get_digest

method is used by the local cache, where we don’t actually need to maintain compliance with any standards about what should go into the digest: https://github.com/pantsbuild/pants/blob/7f257fdf64f11e0064cd6d56a1e1d5d36507ad3f/src/rust/engine/process_execution/src/lib.rs#L976C5-L995 … having said that, that method currently uses the exact same cache entry definition as our other process execution, because it delegates to

make_execute_request

: https://github.com/pantsbuild/pants/blob/7f257fdf64f11e0064cd6d56a1e1d5d36507ad3f/src/rust/engine/process_execution/src/lib.rs#L1065-L1296

make_execute_request

converts a

Process

struct into an

Action

and a

Command

, which are the protobuf/REAPI level definitions of a process: they are used for remote execution and remote caching, and so are standardized

so: to create a new property which is observed for memoization, but not in the cache, you would add a property to

Process

, but then not include it when creating an `Action`/`Command` from a

Process

in

make_execute_request

assuming that caching of failure is disabled both remotely and locally (which it is by our local cache, and also by how we invoke remote processes), then the “skip memoization but still hit the cache” approach could be accomplished that way.

----

alternatively, as you mentioned elsewhere, you could add a

retries

property to

Process

and have that be implemented by a

CommandRunner

layer (similar to our

bounded

,

cache

,

switched

, etc CommandRunners, in this directory). in that approach, you would add a

retries

or

attempts

property to

Process

, and then your

CommandRunner

would retry the process in a loop until it got a success or until it ran out of attempts.

----

i think that the latter might be fairly straightforward to allow for retry of failed processes, and it would expose a very clean API to callers (since they would simply define a property on their

Process

to trigger the retry)

cc @average-vr-56795 in case you have any references/thoughts on how Bazel does process retries

Ok gotcha makes sense I'll bow out

I had some time to do some thinking and some hacking: The root of the problem is that a retry involves submitting the same process to be run. The problem is that because the process is the same, the Runtime Graph sees this node as already executed and returns the (failed) result. There currently isn't a way to prevent the Runtime Graph from storing the execution value of a node (storing the node result is how the result is later retrieved). Uniquifying the Process we send would ensure that Runtime Graph Node is new and would be rerun. The problem is that the node "Run tests (attempt 3 of 4)" is kindof the same as "Run tests (attempt 1 of 4)", so we would want a successful run of all attempts to have the same cache key. And that's where we get the idea that we want to uniquify the Process for the Runtime Graph, but not for any of the inter-run caching. Seems adding a field on Process (Process.attempt) and then just not changing anything in

make_execute_request

just works. An advantage of this is that it allows for the caller implement custom retry logic. For example, the Test goal could rebuild the batches to include only the failed tests. (This may or may not be a good thing. If tests fail because they're concurrency unsafe, rerunning only the failed ones may reduce the concurrency and mask the problem). The disadvantage is that all callers need to have some logic for retrying processes. Check it out here: https://github.com/pantsbuild/pants/pull/19760