Worked on a slightly different approach to Pants bash completions this evening. I was wondering how usable completions that called back into Pants dynamically would be (eliminating the "completions per repo" problem). Turns out, they're incredibly unusable on my machine, because the

pants help-all

time is just too slow. Instead, for those who have

jq

installed, I've tried yet more slightly different. I call

pants help-all

and cache the resulting JSON using the md5 of the pants.toml file. I then check that md5 on completion, and pull the giant json file back in to use

jq

to parse. This won't work for nested

pants

usage, unfortunately. Also, this ends up being too slow to use for parsing all the scoped options from all the keys in the JSON (at least, using a naive for loop approach - I think

jq

has a syntax for what I'm trying to do, but I haven't dug into the manual yet). Either way, for this approach, it's probably worth just doing a one-time parse and caching all the results somewhere, until the pants.toml MD5 changes, and then it would be super fast (rather than re-parsing a huge JSON file all the time). https://gist.github.com/sureshjoshi/cbe50559eb85f21f079e45681d9b486b

Either way, for this approach, it's probably worth just doing a one-time parse and caching all the results somewhere, until the pants.toml MD5 changes, and then it would be super fast (rather than re-parsing a huge JSON file all the time).

Which was the approach taken by the Pants v1 shell completion support. The options config was converted to be shell code and then the shell completion function could just consult shell variables to determine what the completions were.

Ooh, that's interesting - I never knew that. I mean, the hope would be that we could get the load time down - but I think as John mentioned, we'd still need to import the world. Would just be nice to only do that once. My hope would still be complete for a

pants-completion-handler

idea. I think that would be slick, and extensible, and not require any 3rd party tooling

Is there an API the completions stuff could use to query pantsd?

@witty-crayon-22786 ^^ ? I know unfortunately little about the daemon

Is there an API the completions stuff could use to query pantsd?

`pantsd`’s only interface at the moment is CLI invokes. the actual runtime of: 1. starting the native client from https://github.com/pantsbuild/pants/pull/11922 2. connecting to a server … should be sub 50 ms though. most of the overhead when using 11922 is in options parsing and bootstrapping.

`pantsd`’s only interface at the moment is CLI invokes.

It need not be given the Build Server Protocol support demonstrates Pants can function as a request/response server as well.

Well, sure. But that's still using a CLI invoke.

A similar approach to BSP could be taken.

pantsd

listens on a UNIX domain socket under BUILDROOT/.pants.d for requests and then invokes an uncached rule to get the result.

Yea, maybe. I don't expect that adding a new protocol would be faster. The majority of the (unavoidable) latency is in starting the client process, and in options parsing / etc.

well it need not me invoked on every completion request. a better system would have the completion support recognize when it has changed into a new/different Pants repository, query the pantsd for the completions for that repo, and then cache them as shell code once

It's worth noting that I toyed around with an idea (more or less like this), where the completions script was a shim that calls out to the Pants-installed python interpreter, to then run a script which performs completions without the use of

jq

- performance was perfectly adequate. The ideal world for that would be as you described, where

pantsd

(or whatever) automatically writes out the completions, or caches help, whenever the pants.toml changes. I think that was the hope in one of the more recent tickets on this topic. I think back in the day, I also talked about the idea of the

./pants

script auto-sourcing a completions script as well - which was a bit more 🤷 .