https://pantsbuild.org/ logo
Join Slack
Powered by
Question I’m trying to figure out how a ProcessExe...
# development
e
Question I’m trying to figure out how a ProcessExecutionRequest pointer gets materialized on the rust side of the FFI, and then run. As far as I can tell here is how it works: * the python engine via a product_request ends up calling 
Scheduler.execute
with an execution request that was returned from an ffi call. * That in turn ends up calling the rust ffi 
scheduler_execute
method which materializes the scheduler and then calls its internal 
execute
method. * rust scheduler.execute materializes the roots into the type of request it contains. * This is where it gets a bit fuzzy for me. The node is added to the graph and somewhere along the line the it is cast into a NodeKey. That NodeKey has run called on it at some point by the graph and the rest is history. Rubber ducking this has helped but I still have some questions about how the rust side 
Scheduler.execute
knows how to materialize a given root in the roots list to the correct NodeKey enum variant. I see the NodeKey using the From trait to describe the casting, but I still don’t understand how the root contains the information that it is a specific type of request, i.e which into is called. Any help would be appreciated!
w
python objects are passed around as either `Key`s (which are memoized) or 
Values
(which are not)
the RuleGraph that gets built when an engine is initialized already describes which Node types a Node might depend on
"Select" is a Node that is rarely actually stored in the graph, but which is used all the time by calling 
run
directly (which effectively "inlines" the Node, without storing it)
if you do something like 
schedule.product_request(ExecuteProcessRequest, [ExecuteProcessResult(...)])
, the effect is that there is a root Node of that shape, and the actual type of the root will be 
Select
anywhere else in the graph, the 
Select
will be inlined, so it just looks like 
Rule depends on ExecuteProcess
@early-needle-54791: might be worth pairing on this tomorrow if you want
e
This is really helpful thanks.
pairing could be a good idea. Lets see how far I get on this today
Open in Slack
PreviousNext
Powered by