#22483 [call-by-name] A more natural idiom for polymorphism Issue created by benjyw Consider deprecating

in_scope_types

on `@union`s. Instead, we could derive the in-scope types from the signature of a polymorphic rule. # Background ## Polymorphic Dispatch Polymorphic rules allow dispatch based on runtime types of members of a union. In the old call-by-type world, this meant that, given:

@union
class Base:
    ...

class Member1:
    ...

class Member2:
    ...

@rule
async def rule1(input: Member1) -> Result:
    ...


@rule
async def rule2(input: Member2) -> Result:
    ...

and the following union rule registrations:

UnionRule(Base, Member1),
UnionRule(Base, Member2),

Then

await Get(Result, Base, input)

would dispatch to either

rule1

or

rule2

, depending on the type of

input

. Note that

Member1

and

Member2

do not have to be Python subtypes of

Base

, but it is very common that they are. ## In-scope Types Unlike regular function dispatch, `@rule`s can consume `Param`s available at the callsite, even if not provided explicitly. For example, given

@rule
async def do_something(input: Input, other: Other) -> Result:
    ...

Then

await Get(Result, Input, input)

will invoke

do_something(...)

with the

Other

argument provided from callsite context (e.g., if it was a parameter of the calling rule, or if some other rule can create it from parameters available to

do_something(...)

. This complicates matters for polymorphic dispatch, since we need the relevant polymorphic rules to have stable APIs, so that the caller knows which params to provide to `@rule`s it may not know about in advance. This stability is achieved via the

in_scope_types

argument to the

@union

decorator:

@union(in_scope_types=[Other])
class Base:
    ...

This is a contract that polymorphic dispatch will provide the union member, and instances of each of the

in_scope_types

, as params to the callees. # Polymorphic Call-by-name We are currently ]migrating](#21065) the codebase from call-by-type to call-by-name. Polymorphic call-by-name is implemented via a base

@rule

tagged as polymorphic:

@rule(polymorphic=True)
async def base_rule(input: Base) -> Result:
    ...

So that a call

await base_rule(**implicitly({input: Base}))

dispatches according to the runtime type of

input

. # Proposed Idiom Once we are entirely call-by-name, we might consider the following changes, to make Pants rule graph polymorphism follow a more natural idiom: 1. Use subtyping instead of

UnionRule

registration. Today it is already very common for union members to also be Python subclasses of the union type. So we might as well require it, and consult the MRO instead of explicitly registering unions. 2. Use the `base_rule`'s signature as the stable extension API, instead of

in_scope_types

. For example, given

@rule(polymorphic=True)
async def base_rule(input: Base, other: Other) -> Result:
   ...

we can infer that

Base

and

Other

are the params that will be made available to the polymorphic variants. This has the added advantage of being a property of the base rule, and not of the union itself, so that different rules that are polymorphic on the same type can have different in-scope types. This change will require the `base_rule`'s definition to be available where

in_scope_types

are currently consumed, which may require some plumbing. pantsbuild/pants