Hm, we had wanted to add a plugin hook via the Rules API for setting remoting headers. For example, the plugin could make a network request to get an auth token and then update the headers with it But, it looks like we initialize

RemotingOptions

well before the graph session is initialized. Iiuc, that remoting setup is supposed to happen before we can do anything like use the plugin API, as we can't set up things like the Store and CommandRunner without it Does this indeed sound impossible to add?

Very possible to add!

is to move CommandRunner from Core to Session

That makes sense. But that new plugin could not use the actual Rules API, right? Fundamentally, this plugin's purpose is to assist in setting up the remote store + remote command runner. Therefore, it cannot use things like

Process

because the command runner has not yet been set up We could maybe get into a world where remote setup happens after we already set up the local runners, and after this plugin has executed. We re-init it all. But that sounds pretty complex

Yeah, you wouldn't be able to execute processes if you wanted to use this to initialise how we execute processes

Okay, and intrinsics would probably not work either because the Store is not set up properly, right? Iirc,

DownloadFile

uses the

Store

We could work out a way for you to use a local

Store

, for sure

maybe something like this, but s/command runner/store

We could maybe get into a world where remote setup happens after we already set up the local runners, and after this plugin has executed. We re-init it all.

that does sound pretty complex

we should avoid such a big change right now

that is, circumvent the Rules API and use normal Python, right?

(or an API called from the plugin when it initializes if my “plugin life cycles” change is ever taken something we do)

I think that could work. And it would be safe (caching-wise) to do things like use the

requests

library because this is pre-graph/pre-memoization

I do also vague thoughts specifically on auth that ideally there would be a re-callable auth plugin, tightly integrated with the CommandRunner, rather than just an up-front thing, so that it can deal with things like tokens expiring or being revoked mid-build

@average-vr-56795: yes which may happen eventually. we are intentionally punting on that to get Pants using remote caching this week.

Oh yeah, definitely not trying to change priorities, but if we're thinking future design... 🙂

definitely would love your thoughts there!

in the medium term, we might have a “bootstrap” Scheduler that is used for options parsing, and to construct the actual runtime Scheduler. this sounds like another usecase for that. https://github.com/pantsbuild/pants/issues/10360#issuecomment-660351967

So I worked around this problem by not using the Rules API. I added a bootstrap option

--remote-auth-plugin

, which takes the format

path.to.module:func

and invokes it as normal Python That works great, but Toolchain needs to be able to access arbitrary subsystems for its plugin. Not only the bootstrap options. So I think we need a well-formed

Options

object That means passing

Options

instead of

OptionsBootstrapper

to a couple places like

EngineInitializer.setup_graph()

..It looks like that's feasible - we parse options before we setup the graph. But does anyone know of an issue with now needing to wire

Options

everywhere instead of

OptionsBootstrapper

?

We'd need to make it very clear that this function shouldn't generate anything which would alter the outcome of actions

(Because these won't be part of cache keys anywhere)

Right now, the plugin returns this dataclass

@dataclass(frozen=True)
class AuthPluginResult:
    """The return type for a function specified via `--remote-auth-plugin`."""

    store_headers: Dict[str, str]
    execution_headers: Dict[str, str]
    auth_host_name: Optional[str] = None
    expiration: Optional[datetime] = None

Only the first two are actually used. I'm not sure the second will actually make sense, but Asher had suggested them -- this return type is used to set the

store_headers

and

execution_headers

that we send over FFI to setup the

RemotingOptions

. If those change, would our caching not pick that up? Or you mean something else?

The settings used to initialise the CommandRunner are not part of caching anywhere

WIP: https://github.com/pantsbuild/pants/pull/11503 Because this happens pre-memoization, this plugin hook will execute every run