< curved television 6568> ` pants help ApiType` is such a ga Pants #development

<@U01PZK60W2F> `./pants help ApiType` is such a ga...

hundreds-father-404

01/20/2022, 12:45 AM

@curved-television-6568

./pants help ApiType

is such a game changer for plugin development. I've been trying to figure out how a Java API works, and realized I could run

./pants help CoursierResolvedLockfile

. Amazing! Thank you again 🙌

🙌 2

👍 1

busy-vase-39202

01/20/2022, 12:51 AM

Hey, this sounds like subject for a good blog post...

hundreds-father-404

01/20/2022, 12:53 AM

It could be! It'd be interesting because it requires showing how the Plugin API works, which we haven't spent much time talking about beyond Benjy's PyCon talk Also!!! We should rewrite our Rules API docs to make this new mechanism front-and-center. We talk about how it's a type-driven API, and this is the way to see how all the types relate. (Whereas before you would have to manually inspect things, or inspect an enormous generated graph)

bitter-ability-32190

01/20/2022, 1:16 AM

Holy crap!!

bitter-ability-32190

01/20/2022, 1:17 AM

This is amazing!

happy-kitchen-89482

01/20/2022, 1:19 AM

It is!!

hundreds-father-404

01/20/2022, 1:30 AM

Oh cool, we should also probably stop trying to include comments in docstring like "To use this, use `Get(ProcessResult, Process)`". History shows those often become stale, and that info is now auto-generated. Wdyt?

✅ 1

curved-television-6568

01/20/2022, 6:20 AM

Great 😊 I’m currently refactoring that feature, as I realized I made it too rule centric, when it ought to be type centric. This in prep for adding this stuff to the docsite.. (implementation detail, shouldn’t affect the help cli noticeably)

curved-television-6568

01/20/2022, 6:21 AM

@hundreds-father-404 try

./pants help-advanced ..

for even more details ;)

flat-zoo-31952

01/20/2022, 2:29 PM

what version did this land in? I'm not seeing this in 2.9.0rc6 ...

curved-television-6568

01/20/2022, 2:32 PM

yeah, it’s in 2.10.0.dev1

curved-television-6568

01/20/2022, 2:32 PM

so it’s not out in a stable version yet…

flat-zoo-31952

01/20/2022, 2:32 PM

haha okay I'll wait then 😌 ⏲️

👍 1

curved-television-6568

01/20/2022, 2:34 PM

ops, yeah it’s not even in the dev1, it’s on main only still..

hundreds-father-404

01/20/2022, 6:04 PM

Oh hm, @curved-television-6568 I didn't think about namespacing of types. It's possible to define the same class name in two files, where the module is what makes them unique I wonder if we should do: 1. the 100% safe thing of having it be

./pants help pants.engine.process.Process

? Rather than

./pants help Process

. 2. If we have ambiguity, print all matching types. 3. If we have ambiguity, fail eagerly and require plugins rename things 4. If we have ambiguity, choose 1 type to output and ignore the others

hundreds-father-404

01/20/2022, 6:05 PM

I'm thinking #2? #1 is annoying to track down the module name Our UI output is already structured to show multiple entitites at the same time.

./pants help black python_source

works

curved-television-6568

01/20/2022, 6:06 PM

I’m currently refactoring this whole thing (or rather, adding

PluginAPITypeInfo

as a first class thing in the

AllHelpInfo

structure..)

curved-television-6568

01/20/2022, 6:07 PM

and for this, I’ve thought about this naming collission situation.

curved-television-6568

01/20/2022, 6:07 PM

I thought about using the canonical name with module, only when required to resolve ambiguities.

curved-television-6568

01/20/2022, 6:08 PM

and when you want help for a type, if you provide an ambigous option, I think we could present the options as a “did you mean …”

hundreds-father-404

01/20/2022, 6:09 PM

I think we could present the options as a “did you mean …”

I was considering that too, but it would require adding support for

./pants help

being interactive which is a big undertaking and I'm not convinced we want to make it interactive Which led me to the second proposal, just output it all

hundreds-father-404

01/20/2022, 6:09 PM

We probably want to always show the module, regardless of ambiguity. If anything, it helps you discover where things like

Process

are defined

curved-television-6568

01/20/2022, 6:18 PM

… it would require adding support for
./pants help
being interactive

How so? No more interactive than:

Copy code

$ ./pants testa ::
Unknown goal: testa
Did you mean test?

hundreds-father-404

01/20/2022, 6:22 PM

Oh I see, so you rerun. I thought you meant waiting for user input. Cool So, you could either do

./pants help Process

./pants help pants.engine.process.Process

curved-television-6568

01/20/2022, 6:42 PM

exactly.

curved-television-6568

01/20/2022, 6:43 PM

but I think I’ll land an intermediate PR first, without disambiguation, as I suspect it’s going to be a bit large otherwise..

hundreds-father-404

01/20/2022, 6:43 PM

I think we could probably get away with keeping it simple to simply output all matching types. Sounds like less code, fewer things to document, fewer edge cases. But that sounds viable as well

curved-television-6568

01/20/2022, 6:43 PM

yeah, one step at a time.. 😉

curved-television-6568

01/20/2022, 6:45 PM

just to get a feel for what I’m working with…

Copy code

$ jq '.name_to_api_type_info["TestFieldSet"]' all.json 
{
  "consumed_by_rules": [],
  "dependees": [
    "pants.backend.experimental.go",
    "pants.backend.experimental.java",
    "pants.backend.experimental.scala",
    "pants.backend.python",
    "pants.backend.shell"
  ],
  "dependencies": [],
  "documentation": "The fields necessary to run tests on a target.",
  "is_union": true,
  "module": "pants.core.goals.test",
  "name": "TestFieldSet",
  "provider": "pants.core",
  "returned_by_rules": [],
  "union_members": [
    "GoTestFieldSet",
    "JunitTestFieldSet",
    "PythonTestFieldSet",
    "ScalatestTestFieldSet",
    "Shunit2FieldSet"
  ],
  "union_type": null,
  "used_in_rules": [
    "pants.core.goals.test.run_tests"
  ]
}
~/src/github/kaos/pants (api_types_docsite *)
$ jq '.name_to_api_type_info["BuiltPackage"]' all.json 
{
  "consumed_by_rules": [],
  "dependees": [
    "pants.backend.experimental.docker",
    "pants.backend.experimental.go",
    "pants.backend.python",
    "pants.backend.shell",
    "pants.core"
  ],
  "dependencies": [
    "builtins",
    "pants.engine.process",
    "pants.engine.target",
    "pants.engine.unions",
    "pants.option.global_options"
  ],
  "documentation": "BuiltPackage(digest: 'Digest', artifacts: 'tuple[BuiltPackageArtifact, ...]')",
  "is_union": false,
  "module": "pants.core.goals.package",
  "name": "BuiltPackage",
  "provider": "pants.backend.experimental.docker, pants.backend.experimental.go, pants.backend.experimental.java, pants.backend.python, pants.core",
  "returned_by_rules": [
    "pants.backend.docker.goals.package_image.build_docker_image",
    "pants.backend.go.goals.package_binary.package_go_binary",
    "pants.backend.java.package.deploy_jar.package_deploy_jar",
    "pants.backend.python.goals.package_pex_binary.package_pex_binary",
    "pants.backend.python.goals.setup_py.package_python_dist",
    "pants.core.target_types.package_archive_target"
  ],
  "union_members": [],
  "union_type": null,
  "used_in_rules": [
    "pants.backend.docker.goals.run_image.docker_image_run_request",
    "pants.backend.docker.util_rules.docker_build_context.create_docker_build_context",
    "pants.backend.go.goals.run_binary.create_go_binary_run_request",
    "pants.backend.python.util_rules.local_dists.build_local_dists",
    "pants.backend.shell.shell_command.prepare_shell_command_process",
    "pants.core.goals.package.package_asset",
    "pants.core.goals.publish.package_for_publish",
    "pants.core.goals.test.build_runtime_package_dependencies",
    "pants.core.target_types.package_archive_target"
  ]
}

🙌 1

curved-television-6568

01/21/2022, 2:28 PM

OK, so the PR is here https://github.com/pantsbuild/pants/pull/14227 I’m happy with the results (bar some dubious performance) although the PR size is a bit intimidating.

2 Views

Open in Slack

Previous Next