Hey, I have a

MultipleSourcesField

where I just provide the

default

field. However, in

pants peek

no sources are listed, I just see

sources_raw

. Do I need to write custom code to hydrate the sources?

I would think the answer to that is no, so feels like something fishy is up.. what’s the output look like?

This is for a custom Helm plugin that works with our gitops process. Here's the sources_raw from pants peek

"sources_raw": [
      "Chart.yaml",
      "Chart.yml",
      "values.yaml",
      "values.yml",
      "values.*.yaml",
      "values.*.yml",
      "templates/*.yaml",
      "templates/*.yml",
      "ci-state.yaml",
      "ci-state.yml",
      "ci-state.*.yaml",
      "ci-state.*.yml"
    ]

and this is the field

class CustomHelmSourcesField(MultipleSourcesField):
    default = tuple(
        itertools.chain(
            *([f"{filename}.yaml", f"{filename}.yml"] for filename in ["Chart", "values", "values.*", "templates/*", "ci-state", "ci-state.*"])
        )
    )
    expected_file_extensions = (".yaml", ".yml", ".tpl")

huh… mm, trying to see what we’re missing here. in the mean time, can’t help but to suggest a minor improvement to your generators:

tuple(itertools.chain.from_iterable(["name1", "name2"] for filename in [...]))

Nothing out of the ordinary in my target class, in addition to the

CustomHelmSourcesField

there's the common target fields,

Dependencies

, and a few more custom

StringField

types.

do you have any files that the default globs would match for the defined target?

Yes, right now there is a

Chart.yaml

,

values.yaml

, and

values.dev.yaml

given that you get the raw sources field from peek, the target exists properl, but for some reason it seems to not pick them up.

in the same directory as

BUILD

I found the issue

Ah, sorry I didn’t ask that explicitly, was on my mind.

Do I need to create a custom hydrate sources or will something similar to this work? I'm assuming this is union ruled somewhere though it's not in this module.

if you merely need the sources from disk as-is, you don’t need to do anything extra.

I actually need multiple sources fields. How can I get them to populate properly if I have 2?

admit the details are a bit hazy now I start looking at it

That would probably be an acceptable solution, but just to check all the boxes, the built-in

helm_chart

target does have multiple source fields. https://github.com/pantsbuild/pants/blob/7fb39c28ec9a289c35ff289eff95925ec19c574d/src/python/pants/backend/helm/target_types.py#L199-L200

Oh, ok. yea in that case it seems to work, I guess as long as they handle source hydration in that backend then…

will try real quick

@witty-family-13337 may be able to provide more insight there..

Interesting result

sources": [
      "buildbarn/Chart.yaml"
    ],
    "sources_fingerprint": "f1ef2543a76d156d9b705ba40578031e44d580f715b703e1cdaccd47d729195f",
    "sources_raw": [
      "values.yaml",
      "values.yml",
      "templates/*.yaml",
      "templates/*.yml",
      "templates/*.tpl",
      "crds/*.yaml",
      "crds/*.yml"
    ],

it's interesting indeed, I never looked to deep into the behavior of

peek

in the Helm backend so I guess this can be considered an issue. I've been considering refactoring the Helm chart target into smaller targets, both of which with a single sources field, but for a different reason. This could be yet another reason to do so

The main thing I'm trying to get is get the

--changed-since

flag to work correctly.

yeah, I guess that would be an issue with the current implementation too

We use ArgoCD so we don't ever actually install our charts, we just update a ref in a master reference repo and Argo handles the rest. The existing helm backend also tries to fully resolve a chart whenever it sees it, but our base charts are in private registries and it's a huge pain configuring them to pull locally.

fair enough