#18895 Proposal: Introduce rule-name-centric calling convention, and deprecate return-type-centric `Get` Issue created by stuhood tl;dr: This issue proposes introducing a syntax to call `@rule`s by name, and deprecating the

Get

-by-return-type syntax in almost all cases (`@union`s TBD). Motivation Currently, rather than actually referring to the

@rule

that will be called, `Get`s use a return-type-centric syntax (

await Get(SomeReturnType, SomeArgument(..))

). But as our set of plugins has expanded and our understanding of `@union`s has matured, it's becoming clearer that one of the motivations for that syntax is probably no longer valid. "`@rule` graph solving" (documented here) refers to the process of: 1. choosing the

@rule

implementations to use for: 1. each positional argument to a

@rule

2. each of a `@rule`'s `Get`s 2. determining the set of `Param`s which must be part of the runtime graph

Node

for a

@rule

, because they will be used (transitively) to compute its dependencies That second aspect of

@rule

graph solving (determining the set of `Param`s that a

@rule

needs in order to run) is something which cannot practically be done by hand, and so it continues to be an essential component of composing `@rule`s. But the first aspect (deciding which `@rule`s will be called) can definitely be done by hand, since at a fundamental level, it can be reduced to the same sort of function calling convention that is used in most programming languages: calling a function by name. There is no hardship involved in calling functions by name, but when the v2 engine was originally created, particular focus was paid to pluggability and extensibility. Avoiding calling functions by name meant that the implementations of your dependencies could be swapped out without mocking, and that "abstract" `@rule`s (currently represented as `@union`s) would be on equal footing with other calls. But it's become clear in the intervening time that explicitly introducing and tracking pluggability upstream via a set of declared `@union`s is more maintainable and easier to reason about than attempting to swap out `@rule`s. And the cost/complexity of rule graph solving means that: 1.

@rule

graph error messages remain incomprehensible 2. startup time for

pantsd

is noticeable (with enough `@rule`s installed, even in

--release

mode) 3. feature development is inhibited by the inability to make fundamental changes to

@rule

graph solving (e.g. #7654 (comment)) To simplify and remove roughly half of the magic involved in rule graph solving, we should deprecate the

Get

-by-return-type syntax in favor of calling `@rule`s by name (`@union`s TBD). Syntax One syntax for accomplishing this would be to adjust the

@rule

decorator to convert the function it is applied to into a

Callable

with adapted arguments, which returns a

Get

-shaped type. For a

@rule

like:

@rule
def my_rule(arg1: Arg1, arg2: Arg2) -> ReturnType:
    return ReturnType()

Callsites could allow positional arguments before a literal

...

(ellipses). After the ellipses, they would use the existing relevant portion of

Get

syntax to provide any (transitive) parameters for the

@rule

call:

# Equivalent to `Get(ReturnType)` (i.e. no arguments to `Get`). All arguments would be
# computed from Params which were already in scope:
await the_rule_to_call(...)

# Using a positional arg. Roughly equivalent to `Get(ReturnType, Arg1())`: the remaining arguments would be
# computed from Params which were already in scope:
await the_rule_to_call(Arg1(), ...)

# Two positional args. Roughly equivalent to `Get(ReturnType, {Arg1(): Arg1, Arg2(): Arg2)`. Note that
# `@rule` graph solving is still necessary for this case, because the called `@rule` may have `Get`s
# which have additional dependencies.
await the_rule_to_call(Arg1(), Arg2())

# Equivalent to `Get(ReturnType, Arg1())`:
await the_rule_to_call(..., Arg1())

# Equivalent to `Get(ReturnType, {Arg1(): Arg1, Arg2})`:
await the_rule_to_call(..., {Arg1(): Arg1, Arg2(): Arg2})

pantsbuild/pants