[Call-by-name week] Here is the tracking spreadshe...
# development
h
w
There is a good chance that there are some finished backends which I forgot to mark off
e.g. Makeself probably
Also, the only reason a lot of these backends are unfinished is probably because I'm not familiar enough with the underlying technology to debug problems Or, I just don't have the dependencies installed (e.g. Docker)
@happy-kitchen-89482 Since you've got your name on a couple docker items - did you want to grab docker.trivy (4 GET calls), and podman (not sure if there is any code in the backend)
h
Yep, I’ll take em
👍 1
f
Bugs encountered while migrating: • The migration script failed to add an import for
implicitly
when run against the JVM protobuf codegen backends. • The migration script failed to put a
Process
inside
implicitly
for
fallible_to_exec_result_or_raise
resulting in rule graph errors (in Java codegen protobuf backend). • If the migration script is applied to a file again, some call-by-name usages are renamed unnecessarily, e.g.
get_source_root
to
get_source_root_get
• For the docs, the backends to be migrated must be activated in
pants.toml
. If there is an "easy" fix for any of these, would love a pointer to where to fix.
Thought re reducing use of implicitly: We should probably move the "request" type for a call-by-name to be the first positional argument so it does not need to be passed via
implicitly
. This may still require
implicitly
to be present of there are other args, but should still look nicer. Thoughts?
Also, a little annoying to have
transitive_targets
rule always renamed to
transitive_targets_get
since most uses assign result to
transitive_targets
Maybe we should rename the
transitive_targets
rule to have a verb in it?
💯 1
bug in the rule visitor:
Copy code
pants.base.exceptions.RuleTypeError: /Users/tdyas/Projects/Pants/pants/src/python/pants/backend/kotlin/compile/kotlinc_plugins.py:138: Could not resolve type for `artifact_field.to_address_input` in module pants.backend.kotlin.compile.kotlinc_plugins.
w
> We should probably move the "request" type for a call-by-name to be the > first positional argument so it does not need to be passed via
implicitly
> I'm surprised it sometimes isn't like that - seems like the most intuitive when making a function, but at the same time, it didnt really "matter" in the Get syntax
For the docs, the backends to be migrated must be activated in
pants.toml
.
Yeah, I updated the tracking ticket with an example - as I think it was easy to gloss over the comment
I should note that during the migration, I tried to avoid changing functions/parameters as much as possible - to reduce the burden on the reviewers. The call by name code can be re-purposed into a linter to eventually look for and find spurious implicitly's
👍 1
f
Copy code
E       native_engine.IntrinsicError: Call(pants.backend.codegen.protobuf.scala.rules.materialize_jvm_plugins, MaterializedJvmPlugins) was not detected in your @rule body at rule compile time.
If one rule references another rule via call-by-name in the same file, then the referenced rule apparently must be defined first, or else this error results.
w
Ahhh, yes, this was the original problem with the rule_helper and stuff - one of the implict "rules" of using a
rule_helper
. I saw the rule, but didn't realize the location was below
f
Also, the rule visitor could do better at figuring out the type. Had to fix an error this way by adding a type annotation:
Copy code
--- a/src/python/pants/backend/kotlin/compile/kotlinc_plugins.py
+++ b/src/python/pants/backend/kotlin/compile/kotlinc_plugins.py
@@ -136,7 +136,8 @@ async def resolve_kotlinc_plugins_for_target(
         plugin[KotlincPluginArtifactField].to_address_input() for plugin in candidate_plugins
     ]
     artifact_addresses = await concurrently(
-        resolve_address(**implicitly(address_input)) for address_input in address_inputs
+        resolve_address(**implicitly({address_input: AddressInput}))
+        for address_input in address_inputs
     )
     candidate_artifacts = await resolve_targets(**implicitly(Addresses(artifact_addresses)))
Even though the
address_inputs
is explicitly typed as
list[AddressInput]
https://github.com/pantsbuild/pants/pull/22263 is blocking the scala protobuf backend migration if anybody can take a look
w
I'm offline until about 8:30pm EST - I took a quick look, but didn't have enough context offhand to green light it
f
Benjy approved it, all good.
separate question for any one around: what is the call-by-name syntax for invoking a
UnionRule
where multiple plugins could provide rules?
f
These three don't need to be migrated, if someone wants to check those off
Copy code
pants.backend.experimental.go
pants.backend.experimental.go.debug_goals
pants.backend.experimental.java.debug_goals
Actually, maybe I'm mistaken here. Running
pants migrate-call-by-name src/python/pants/backend/experimental/go/::
returns
None of the 12 requested files are part of the 197 files in the migration plan.
But there are files that are imported by that
register.py
, they just exist in
pants.backend.go
Should I migrate the entire go backend?
w
Are they in the pants.toml?
f
pants.backend.experimental.go
is. I guess thats the primary go backend though, I don't see one called
pants.backend.go
w
Hmmmmmm
Ah, right... How do those get migrated. I had a few small ones, so I did them manually. Are you able to put this one in pants and migrate it?
pants.backend.experimental.go.debug_goals
f
Theres nothing really in there, just imports from
pants.backend.go.goals
and
pants.backend.experimental.go.register
w
Oh, sweet
The backends/go/goals/debug_goals has a bunch of Gets. Is that different from the registered one?
f
Ok, enabled the backend and ran it just on that file
Copy code
pants migrate-call-by-name src/python/pants/backend/go/goals/debug_goals.py
Running over the actual location of the register.py didn't work. ie
pants migrate-call-by-name src/python/pants/backend/experimental/go/debug_goals/::
w
Oh, yeah, the experimental register.py is just to "mark" the backend as experimental. All the real code is in the non-experimental path.
... I never thought about how unintuitive that is
f
Sometimes there is code in the experimental path too
w
I don't know that I've ever seen that - but it was my understanding that it should "usually" just be a register.py - but good point, if someone registered rules there - it would need to be migrated there too