Question I’m trying to figure out how a ProcessExe...
# development
Question I’m trying to figure out how a ProcessExecutionRequest pointer gets materialized on the rust side of the FFI, and then run. As far as I can tell here is how it works: * the python engine via a product_request ends up calling
with an execution request that was returned from an ffi call. * That in turn ends up calling the rust ffi
method which materializes the scheduler and then calls its internal
method. * rust scheduler.execute materializes the roots into the type of request it contains. * This is where it gets a bit fuzzy for me. The node is added to the graph and somewhere along the line the it is cast into a NodeKey. That NodeKey has run called on it at some point by the graph and the rest is history. Rubber ducking this has helped but I still have some questions about how the rust side
knows how to materialize a given root in the roots list to the correct NodeKey enum variant. I see the NodeKey using the From trait to describe the casting, but I still don’t understand how the root contains the information that it is a specific type of request, i.e which into is called. Any help would be appreciated!
python objects are passed around as either `Key`s (which are memoized) or
(which are not)
the RuleGraph that gets built when an engine is initialized already describes which Node types a Node might depend on
"Select" is a Node that is rarely actually stored in the graph, but which is used all the time by calling
directly (which effectively "inlines" the Node, without storing it)
if you do something like
schedule.product_request(ExecuteProcessRequest, [ExecuteProcessResult(...)])
, the effect is that there is a root Node of that shape, and the actual type of the root will be
anywhere else in the graph, the
will be inlined, so it just looks like
Rule depends on ExecuteProcess
@early-needle-54791: might be worth pairing on this tomorrow if you want
This is really helpful thanks.
pairing could be a good idea. Lets see how far I get on this today