Hi I m attempting to implement a couple of targets rules tha Pants #plugins

Hi, I'm attempting to implement a couple of targe...

cold-mechanic-10814

05/12/2025, 1:13 PM

Hi, I'm attempting to implement a couple of targets/rules that implement basic codegen (generate a JSON file from a YAML source, etc.). I've got the basics working, but I'm not sure how I can "chain" the codegen targets. Here's an illustrative BUILD file:

Copy code

yaml_file(
    name="my_yaml",
    source=http_source(
        "<https://raw.githubusercontent.com/codefresh-io/yaml-examples/refs/heads/master/codefresh-build-1.yml>",
        len=197,
        sha256="4f0f073a576fc44d1ad670bf886fb24f394f63b6eabb7677eb197d427f5db7b0",
    ),
    convert_to_json=True
)

json_file(
    name="my_json",
    source="codefresh-build-1.json",
    dependencies=[":my_yaml"]
)

The

yaml_file

target works fine. The rule is of the form

Copy code

@rule
async def generate_yaml_from_yaml_source(
    request: GenerateYAMLSourcesRequest,
) -> GeneratedSources:

Where

GenerateYAMLSourcesRequest

is subclass of GenerateSourcesRequest
. It's when I try to trigger the generation for the

json_file

target that the issue occurs. The target is also backed by a rule based on a subclass of GenerateSourcesRequest
, and in addition has code to resolve the dependencies and provide them as a snapshot during the generation. The code fails, seemingly before my rule is triggered, with the following error:

Copy code

native_engine.IntrinsicError: Unmatched glob from src/python/libs/portland/connectors/my_connector:my_json's `source` field: "src/python/libs/portland/connectors/my_connector/codefresh-build-1.json"

What's the appropriate way to chain these codegen steps - with the caveat that the

json_file

target won't always be used in a chain, it should also be used independently.

fast-nail-55400

05/12/2025, 4:00 PM

Maybe remove the

source

field from the

json_file

target type?

fast-nail-55400

05/12/2025, 4:00 PM

That field is used when there is an actual source file in the repository.

fast-nail-55400

05/12/2025, 4:01 PM

There is no

codefresh-build-1.json

in the repository, correct? If so, the "Unmatched glob" error makes sense since technically the

source

field didn't match a file in the repository.

fast-nail-55400

05/12/2025, 4:02 PM

How did you define the

GenerateSourcesRequest

for the

json_file

target type?

cold-mechanic-10814

05/12/2025, 8:37 PM

Thanks for your reply - if I remove the source field, will the target still be picked up by codegen? You're correct that the

codefresh-build-1.json

isn't materialized in the repository. It's an output of the

yaml_file

target. The request looks like this:

Copy code

class GenerateJSONFileSourceRequest(GenerateSourcesRequest):
    input = JSONFileSourceField
    output = FileSourceField

Where

JSONFileSourceField

is a subclass of

AssetSourceField

. I tried making it a subclass of

FileSourceField

, but pants threw an exception about ambiguous generation targets

cold-mechanic-10814

05/14/2025, 1:52 PM

In the end I managed to find a solution by adding an optional source field that is never used – it just exists so that the target is compatible with HydrateSources. The target has other fields that are SpecialCaseDependencies that are used to chain the codegens

👍 1

3 Views

Open in Slack

Previous Next