Hi everyone. Is there by any chance an updated version of https://www.pantsbuild.org/stable/docs/writing-plugins/common-plugin-tasks/add-codegen anywhere? I'm having some issues, and at least part of it seems to be outdated class names in the doc.

My general goto is to either reference one of my known good rules; or one of the built-in plugins, unfortunately. With that said, I haven't noticed any previously working code in my plugins stop working around codegen. As an example; this is how my union rule is set up:

class GenerateKubernetesFromKustomizeRequest(GenerateSourcesRequest):
    input = KustomizeSourcesField
    output = KubernetesSourceField

def rules():
    return [
        *collect_rules(),
        UnionRule(GenerateSourcesRequest, GenerateKubernetesFromKustomizeRequest),
    ]

And then the rule itself is declared like this:

@rule
async def generate_kubernetes_from_kustomize(
    request: GenerateKubernetesFromKustomizeRequest,
    kustomize: KustomizeTool,
    platform: Platform,
) -> GeneratedSources:

Which has also been unchanged for quite a while.

hmm. I can make the error go away by changing the parameters (like removing the tool), but then it doesn't show up as a valid codegen target. I'll try stubbing out the rule and see what happens.

That is interesting. So if I understand correctly and using my code as an example; removing

KustomizeTool

and all uses thereof in the function makes Pants work but it's not detected as a codegenable-target?

correct. Using the term "work" lightly, since it doesn't actually build my code. But that makes it not give the graph errors.

It seems to me that something is wrong with your tool then, as a starting point. Or somewhere you use it. But that's likely not the cause of the build not working.

certainly possible. This is my introduction to pants, so I'm kind of coding by "guess & check". I tried using Cursor to help, but it got very confused.

So e.g. you have f.ex. a Python package and you want that to be your tool?

a perl script in this case, but yes.

Ok. Is embedding the perl script into your plugin code an option? That'll make your life easier.

Not really. I guess I could copy it somewhere and make it a BinaryPath?

Hmm. The general way I'd do something like this is just manifest the code as a file and add to the sandbox sources when I invoke it. This is how most Pants-embedded scripts do it as well. This is a bit uglier, but it's an example I know from the top of my head: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/providers/python_build_standalone/rules.py#L563

So I did get this to work as a new goal and then I just pulled the script in as if it were one of the source files and ran it with

Process

and that was fine, but I figured I need to switch over to a proper codegen setup to be able to chain my rules.

I'm not sure I know enough at this point to say whether that is a true assertion or not. In general, yes. Doing more with the "right tools" that Pants gives you will make it integrate better and make it easier to expand. But I don't know what your goal is... if it was related to getting the perl script somewhere it sounds like the wrong tool for the job.

The perl script is a code generator - it takes in human written source code and spits out code in a different language. My ultimate goal is to go from the human written code, through a chain of dynamically discovered dependencies, a few different code generators (depending on what dependencies were found), and ultimately into a compiler. The binary generated by the compiler is the actual build product that the users want.

So that does sound like something you'd do with codegen rules, indeed. I think you should be fine with just that. You can see exactly what it looks for here: https://github.com/pantsbuild/pants/blob/b68fbe7784df3862d26835556284efdcbe8ae9b5/src/python/pants/backend/codegen/export_codegen_goal.py#L40 The

export-codegen

goal in general is a gods-end for testing codegen rules, since you can validate in vitro if your rule works

Yes, I've been using

export-codegen

. I was looking at that exact code yesterday... I wish there was a debug print of

all_generate_request_types

and

inputs_to_outputs

.

fd export_codegen_goal.py ~/.cache/nce/

- should be able to modify the code if you can figure out which file matches the pants you're using. Or just run pants from source.

ok. That gives me something to try.