https://pantsbuild.org/ logo
#general
Title
# general
c

cold-soccer-63228

05/17/2022, 3:12 PM
When subclassing
InferDependenciesRequest
, what is the difference between
from_sources
and
infer_from
? The docstring description and example for https://github.com/pantsbuild/pants/blob/c73b9f68b7e05ac455057142ae50c75f2560d06b/src/python/pants/engine/target.py#L2529-L2559 seem to contradict each other
āœ… 1
b

bitter-ability-32190

05/17/2022, 3:17 PM
It seems to me
infer_from
is the class variable you set to the field type.
soruces_field
is the instance variable you'll get which is an instance of the
infer_from
type?
šŸ™Œ 1
c

cold-soccer-63228

05/17/2022, 3:20 PM
Thanks. In the same example,
FortranSources
is referenced. What does such a class look like? I want to be able to infer dependencies based on
*.graphql
files, and want to write something for
GraphQLSources
b

bitter-ability-32190

05/17/2022, 3:21 PM
My advice is grep the Pants codebase for examples: https://github.com/pantsbuild/pants/search?q=InferDependenciesRequest
c

cold-soccer-63228

05/17/2022, 3:23 PM
This is helpful. Thanks!
šŸ™Œ 1
āœ… 1
b

bitter-ability-32190

05/17/2022, 3:23 PM
You'll need a
GraphQLSources
target type. See
PythonSource
or
ScalaSource
target types for example. The target type will have a
source
field of a similar name to your sources. So check out
PythonSourceField
and
ScalaSourceField
for reference. ... Then when you're ready, wire up the "plural" version ("sources")
Feel free to reach out for help ā¤ļø
ā™„ļø 3
c

cold-soccer-63228

05/17/2022, 3:44 PM
Okay, so I'm wondering what I have to do next to get these dependencies appearing in
BUILD
files when I run
./pants tailor
. Here's what I have so far.
Copy code
class GraphQLSources(MultipleSourcesField):
    expected_file_extensions: ClassVar = (".graphql",)


class GraphQLSourceField(SingleSourceField):
    expected_file_extensions: ClassVar = (".graphql",)


class InferGraphQLDependencies(InferDependenciesRequest):
    infer_from = GraphQLSources


def rules() -> List[UnionRule]:
    return [
        UnionRule(InferDependenciesRequest, InferGraphQLDependencies)
    ]
I need to add something to a
register.py
, right?
b

bitter-ability-32190

05/17/2022, 3:45 PM
register
needs to have a
rules()
which returns all of the rules you've authored and
target_types()
returning the target types
Conventionally speaking, you'd put the dep inference rules in a different file/folder, FYI. Feel free to grok the Pants codebase and try and mimic
(but ultimately the engine doesn't care how you structure your code šŸ˜‰ )
c

cold-soccer-63228

05/17/2022, 3:49 PM
Got it, thanks! After I go ahead and add the
rules()
and
target_types()
functions to
register.py
, what else do I need to do to expect
BUILD
files next to
*.graphql
files to be updated with the proper dependencies after running
./pants tailor
? I defined
target_types()
as follows.
Copy code
def target_types() -> List[SourcesField]:
    return [GraphQLSourceField, GraphQLSources]
b

bitter-ability-32190

05/17/2022, 3:56 PM
If you want
tailor
to work, there's work to be done to teach
tailor
about your targets. This is beyond what I've done (we don't use
tailor
quite yet)
Try looking at the code that supports
PutativeTargetsRequest
https://github.com/pantsbuild/pants/search?q=PutativeTargetsRequest
FWIW, is there a reason you need targets for your GraphQL files? Are you running
fmt
or
lint
tools on them?
You might just want to declare them as `resource`s. https://www.pantsbuild.org/docs/assets Python support has a special feature where if enabled, it'll parse strings in youre file and try to infer dependencies on `files`/`resoureces`. So there's a good chance you won't have to declare the dep. See https://www.pantsbuild.org/docs/reference-python-infer
c

cold-soccer-63228

05/17/2022, 4:01 PM
I'm using
ariadne.load_schema_from_path
, which attempts to load
*.graphql
files from within Python code. If I don't declare my GraphQL dependencies, then this ends up not working šŸ˜• https://pantsbuild.slack.com/archives/C046T6T9U/p1652796498287969?thread_ts=1652414328.320049&cid=C046T6T9U
b

bitter-ability-32190

05/17/2022, 4:01 PM
Ah, then 95% sure you just want to use
resources
and then add the dep
c

cold-soccer-63228

05/17/2022, 4:01 PM
It'd be nice if tailor could autogenerate this for me though, no? šŸ˜›
b

bitter-ability-32190

05/17/2022, 4:02 PM
Well, rather than making new target types, you can have tailor declare the
resources
, and implement an
InferDependencies
between your Python code and the GraphQL files whne necessary. It's the least amount of code, and metadata šŸ˜‰
c

cold-soccer-63228

05/17/2022, 4:04 PM
How do I have
tailor
declare the resources for me? The GraphQL paths being imported are variables, not string literals. I would assume that
--python-infer-assets
isn't smart enough to figure it out?
b

bitter-ability-32190

05/17/2022, 4:04 PM
How do I have
tailor
declare the resources for me?
You'll have to grok the
PutativeTargetsRequest
code in the Pants codebase on how that works. Or I can ping another maintainer
The GraphQL paths being imported are variables, not string literals. I would assume that
--python-infer-assets
isn't smart enough to figure it out?
Yeah the inference is very naive today (and likely will always be "naive" to some degree). I thinking writing your own
InferDependencies
plugin is likely the way forward for you.
c

cold-soccer-63228

05/17/2022, 4:08 PM
Thanks for these pointers. Just to help me understand better, I have a few more questions: ā€¢ What exactly is
PutativeTargetsRequest
? How does it differ from
InferDependenciesRequest
at a feature level? ā€¢ When you mention implementing an
InferDependencies
between my Python code and the GraphQL files, are you referring to a subclass of
InferDependenciesRequest
? In that subclass, I would look to see whenever I need
ariadne.load_schema_from_path
, and then if I do, I add all GraphQL resources as dependencies?
b

bitter-ability-32190

05/17/2022, 4:10 PM
What exactly is
PutativeTargetsRequest
? How does it differ from
InferDependenciesRequest
at a feature level?
It's the type that powers
tailor
.
InferDependenciesRequest
powers the magical Pants dependency inference for you. To put it in more concrete terms: ā€¢
PutativeTargetsRequest
would be rules that allow
tailor
to generate
python_sources
/`python_tests` etc... ā€¢
InferDependenciesRequest
would be the rules which parse your source code to find imports, then infer the dependencies on your first- and third-party targets
šŸ™ 1
When you mention implementing an InferDependencies between my Python code and the GraphQL files, are you referring to a subclass of InferDependenciesRequest? In that subclass, I would look to see whenever I need ariadne.load_schema_from_path, and then if I do, I add all GraphQL resources as dependencies?
You can grok the codebase on other
InferDependencies
requests and how they're all plugged in. Ack that admittedly, Pants' own codebase is powerful, which also means complex too. High-level: ā€¢ Subclass the class, declare a rule which takes it as a param (put all the juicy code in the rule) and then declare/register a
UnionRule(...)
ā€¢ You probably also want to have your rule take in
AllAssetTargets
as a param. That instance contains all the assets in the repo (files and resources) ā€¢ The rule body should look for the relevant code (in this case
ariadne.load_schema_from_path
) and try and match the argument(s) to declared resources ā—¦ Up to you whether to silently fail if none are found or error.
Correction, you should ask for
AllAssetTargetsByPath
, which already has the assets mapped to their path!
Feel free to post your code if you want pointers or get stuck šŸ™‚
c

cold-soccer-63228

05/17/2022, 4:29 PM
So I seem to be getting a
TypeError
when I run
./pants tailor
now...
Copy code
TypeError: Every element of the entrypoint `target_types` must be a subclass of Target. Bad elements: [<class 'front_porch.modules.peach.internals.pants.graphql.target_types.GraphQLSourceField'>, <class 'front_porch.modules.peach.internals.pants.graphql.target_types.GraphQLSources'>]
Here's what my source looks like.
Copy code
from typing import ClassVar

from pants.engine.target import MultipleSourcesField, SingleSourceField


class GraphQLSources(MultipleSourcesField):
    expected_file_extensions: ClassVar = (".graphql",)


class GraphQLSourceField(SingleSourceField):
    expected_file_extensions: ClassVar = (".graphql",)
Do you know what this is?
b

bitter-ability-32190

05/17/2022, 4:30 PM
You're declaring your fields as target types. target types are types which derive from
Target
So,
GraphQLSources
should be renamed to
GraphQLSourcesField
, and then you need to declare the types
GraphQLSources
which uses the multiple sources field and also declare
GraphQLSource
which uses the single source field. And then also declare the generator from
..Sources
to
..Source
.
However, I still think the earlier point of just using
resources
is the way to go šŸ˜‰
šŸ™ 1
(You won't need all this Target/Field stuff)
šŸ™ 1
c

cold-soccer-63228

05/17/2022, 4:34 PM
Right now, I have a
pants-plugins/graphql/goals/tailor.py
that looks as follows.
Copy code
@dataclass(frozen=True)
class PutativeGraphQLTargetsRequest(PutativeTargetsRequest):
    pass


@rule(level=LogLevel.DEBUG, desc="Determine candidate GraphQL targets to create")
async def find_putative_targets(
    req: PutativeGraphQLTargetsRequest, all_owned_sources: AllOwnedSources
) -> PutativeTargets:
    all_graphql_files = await Get(Paths, PathGlobs, req.search_paths.path_globs("*.graphql"))
    unowned_graphql_files = set(all_graphql_files.files) - set(all_owned_sources)
    pts = [
        PutativeTarget.for_target_type(GraphQLSources, path=dirname, name=None, triggering_sources=sorted(filenames))
        for dirname, filenames in group_by_dir(unowned_graphql_files).items()
    ]
    return PutativeTargets(pts)


def rules() -> Iterable[Rule]:
    return [*collect_rules(), UnionRule(PutativeTargetsRequest, PutativeGraphQLTargetsRequest)]
Instead of the following:
Copy code
pts = [
    PutativeTarget.for_target_type(GraphQLSources, path=dirname, name=None, triggering_sources=sorted(filenames))
    for dirname, filenames in group_by_dir(unowned_graphql_files).items()
]
I should be making it so that it defines a
resource
for each filename, yes? What is the Pants API class for the target that generates a
resource
?
b

bitter-ability-32190

05/17/2022, 4:35 PM
PutativeTarget.for_target_type(ResourceSources
I think.
c

cold-soccer-63228

05/17/2022, 4:50 PM
Ahh, I found it. It was
pants.core.target_types.ResourcesGeneratorTarget
b

bitter-ability-32190

05/17/2022, 4:51 PM
Gahh I think we goofed our naming scheme for a couple of classes šŸ™ˆ
byproduct of code evolution
c

cold-soccer-63228

05/17/2022, 4:59 PM
Haha no worries šŸ™‚ So now, my code looks like this.
Copy code
@dataclass(frozen=True)
class PutativeGraphQLTargetsRequest(PutativeTargetsRequest):
    pass


@rule(level=LogLevel.DEBUG, desc="Determine candidate GraphQL targets to create")
async def find_putative_targets(
    req: PutativeGraphQLTargetsRequest, all_owned_sources: AllOwnedSources
) -> PutativeTargets:
    all_graphql_files = await Get(Paths, PathGlobs, req.search_paths.path_globs("*.graphql"))
    unowned_graphql_files = set(all_graphql_files.files) - set(all_owned_sources)
    pts = [
        PutativeTarget.for_target_type(
            ResourcesGeneratorTarget, path=dirname, name=None, triggering_sources=sorted(filenames)
        )
        for dirname, filenames in group_by_dir(unowned_graphql_files).items()
    ]
    return PutativeTargets(pts)
I'm running
./pants tailor
again, and seeing a different error.
Copy code
12:55:04.61 [ERROR] Encountered 9 rule graph errors:
  No installed rules return the type PutativePythonTargetsRequest, and it was not provided by potential callers of @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]).
    If that type should be computed by a rule, ensure that that rule is installed.
    If it should be provided by a caller, ensure that it is included in any relevant Query or Get.
  No source of dependency AllOwnedSources for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency Get(DigestContents, PathGlobs) for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency Get(Paths, PathGlobs) for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest) for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency Get(SourceRootsResult, SourceRootsRequest) for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency Get(UnexpandedTargets, AddressSpecs) for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency GlobalOptions for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
  No source of dependency PythonSetup for @rule(pants.backend.python.goals.tailor:88:find_putative_targets(PutativePythonTargetsRequest, AllOwnedSources, PythonSetup, GlobalOptions) -> PutativeTargets, gets=[Get(Paths, PathGlobs), Get(DigestContents, PathGlobs), Get(SourceRootsResult, SourceRootsRequest), Get(UnexpandedTargets, AddressSpecs), Get(ResolvedPexEntryPoint, ResolvePexEntryPointRequest)]). All potential sources were eliminated: []
Do I need to add my function definition to a list of rules somewhere? I was following https://github.com/pantsbuild/pants/blob/fda1838ced4b46346d0ee1a8154bbc0de076d7f7/src/python/pants/backend/codegen/protobuf/tailor.py as a reference, which doesn't seem to do that?...
b

bitter-ability-32190

05/17/2022, 5:13 PM
Our rule graph errors are admittedly nasty, sorry about that šŸ˜ž
Usually you haven't registered some rule
c

cold-soccer-63228

05/17/2022, 5:33 PM
Okay, so a few things I tried:
Copy code
def rules() -> Iterable[Rule]:
    return [
        *collect_rules(), 
        find_putative_targets, 
        UnionRule(
            PutativeTargetsRequest, 
            PutativeGraphQLTargetsRequest,
        ),
    ]
This causes the following error:
Copy code
13:28:32.30 [ERROR] 1 Exception encountered:

  AssertionError: A target of type ResourcesGeneratorTarget was proposed at address front_porch/modules/delinquency/loss_mitigation/graphql:graphql with explicit sources loss_mitigation_schema.graphql, but this target type does not have a `sources` field.
I also tried removing the
UnionRule
altogether, which made
./pants tailor
run, but I didn't see anything generated for GraphQL resources in the
BUILD
files I wanted. So I'm guessing I do want the
UnionRule
there. The question then is: "what is this error telling me, and how do I fix it?" Going a bit more high-level, how does
UnionRule
work, exactly? I tried reading https://www.pantsbuild.org/docs/rules-api-unions but I don't think I quite understand...
b

bitter-ability-32190

05/17/2022, 5:37 PM
(sorry was AFK)
c

cold-soccer-63228

05/17/2022, 5:39 PM
No worries!
b

bitter-ability-32190

05/17/2022, 5:39 PM
so
collect_rules
magically finds all the
@rule
functions in your file, so you shouldnt need to list them out. ---
So I'm guessing I do want the
UnionRule
there. The question then is: "what is this error telling me, and how do I fix it?"
It's Pants' way of "injecting" code into the engine. E.g. the engine will ask something "please give me all the implementations of things that consume a
PutativeTargetsRequest
. The engine hands the registered union rules over and they get called
c

cold-soccer-63228

05/17/2022, 5:40 PM
I went ahead and tried printing
sorted(filenames)
, and it seems that they're valid, and I'm passing them into the
triggering_sources
parameter. So I'm surprised that it's telling me these targets don't have a
sources
field, unless I populate that field differently?
b

bitter-ability-32190

05/17/2022, 5:41 PM
If you're past the rule graph error then you're getting close!
Ah sorry I didn't check you earlier.
ResourcesGeneratorTarget
is the wrong type
maybe? Sorry haven't plugged into
tailor
, personally. let me sniff around
c

cold-soccer-63228

05/17/2022, 5:45 PM
Ah, maybe just
ResourceTarget
?
ResourceTarget
also gives the same error
b

bitter-ability-32190

05/17/2022, 5:45 PM
Well if it's a singular file, then probably
resource
. But if there's going to be multiple, use a glob + the generator target.
Which, looks right šŸ¤Æ
c

cold-soccer-63228

05/17/2022, 5:46 PM
šŸ˜­
Copy code
default_sources = default_sources_for_target_type(target_type)
if (explicit_sources or triggering_sources) and not default_sources:
    raise AssertionError(
        f"A target of type {target_type.__name__} was proposed at "
        f"address {path}:{name} with explicit sources {', '.join(explicit_sources or triggering_sources)}, "
        "but this target type does not have a `sources` field."
    )
This seems to be the source of the error
So
default_sources_for_target_type(...)
is returning
tuple()
?...
b

bitter-ability-32190

05/17/2022, 5:48 PM
Copy code
if issubclass(field, MultipleSourcesField):
            return field.default or tuple()
should be firing
c

cold-soccer-63228

05/17/2022, 5:48 PM
Yup, looks like it
Do I need to define something like
GraphQLResourceTarget(ResourceTarget)
, and then override the default fields or something?
b

bitter-ability-32190

05/17/2022, 5:49 PM
Gonna shoot the flaregun. @hundreds-father-404 sorry for long thread. TL;DR see the error
Copy code
AssertionError: A target of type ResourcesGeneratorTarget was proposed at address front_porch/modules/delinquency/loss_mitigation/graphql:graphql with explicit sources loss_mitigation_schema.graphql, but this target type does not have a `sources` field.
Stemming from
Copy code
PutativeTarget.for_target_type(
            ResourcesGeneratorTarget, path=dirname, name=None, triggering_sources=sorted(filenames)
        )
šŸŽ‰ 1
šŸ”« 1
c

cold-soccer-63228

05/17/2022, 6:00 PM
Hmm... I see that
ResourceSourceField
is one of the
core_fields
in the
ResourceTarget
class. And
ResourceSourceField
is indeed a subclass of
SourcesField
. But unfortunately,
SourcesField.default = None
, and it is not overridden. So I'm guessing I need to override
ResourceTarget
with my own
GraphQLResourceTarget
and perhaps do the following?
Copy code
core_fields = (*COMMON_TARGET_FIELDS, Dependencies, GraphQLResourceSourceField)
Then, I would define
GraphQLResourceSourceField.default
to be something that's not
None
. Does this sound right?
šŸ¤” 1
b

bitter-ability-32190

05/17/2022, 6:01 PM
Ohhhhh good eye.
if (explicit_sources or triggering_sources) and not default_sources
You are calling this somewhere with an empty
triggering_sources
šŸ˜‰
@hundreds-father-404 nevermind šŸ˜›
c

cold-soccer-63228

05/17/2022, 6:01 PM
Actually, reading the code that happens in the "happy path", i.e. if the assertion is not tripped, I'm actually thinking this may be a bug
Because we do the following:
Copy code
owned_sources = explicit_sources or default_sources or tuple()
It should be enough to have
explicit_sources
without necessarily having
default_sources
defined
b

bitter-ability-32190

05/17/2022, 6:02 PM
AH good point I think it should be
if not (explicit_sources or triggering_sources or default_sources)
hmm?
šŸ‘ 1
c

cold-soccer-63228

05/17/2022, 6:02 PM
You are calling this somewhere with an empty
triggering_sources
I don't think so.
triggering_sources
is never empty
āœ… 1
b

bitter-ability-32190

05/17/2022, 6:03 PM
File an issue. Sorry about that!
c

cold-soccer-63228

05/17/2022, 6:03 PM
Could I submit a PR directly?
b

bitter-ability-32190

05/17/2022, 6:03 PM
That too šŸ™‚
h

hundreds-father-404

05/17/2022, 6:03 PM
in mtg, but I chagned this code path last week I think. seemed broken before. so check pants version + git history
āœ… 2
c

cold-soccer-63228

05/17/2022, 6:03 PM
Cool. What should the recommended workaround be in the meantime?
b

bitter-ability-32190

05/17/2022, 6:05 PM
Honestly? Try monkeypatching
ResourceSourceField
before calling that method? Or try as you said maybe? Really since this is just to support
tailor
, you can fudge the code a smidgen
You could construct the
PutativeTarget
yourself, copying most of the code from
PutativeTarget.for_target_type
āœ… 1
Yeah probably do that?
āœ… 1
c

cold-soccer-63228

05/17/2022, 6:11 PM
Separately, do you think a GraphQL backend with
tailor
implemented would be useful to have in the
pants
repository itself? All this would do is generate resources wherever
*.graphql
files are found...
b

bitter-ability-32190

05/17/2022, 6:22 PM
I would expect a new backend to do more than support GraphQL resource in build files. If there were formatters / linters then quite possibly
šŸ‘ 2
c

cold-soccer-63228

05/17/2022, 7:55 PM
Don't think I can easily reconstruct
PutativeTarget
šŸ˜•
Copy code
TypeError: Invalid Get. The third argument `FixedPutativeTarget(path='front_porch/modules/activities/graphql', name='graphql0', type_alias='resource', triggering_sources=('activities_schema.graphql',), owned_sources=(), addressable=True, kwargs=FrozenDict({}), comments=())` must have the exact same type as the second argument, <class 'pants.core.goals.tailor.PutativeTarget'>, but had the type <class 'front_porch.modules.peach.internals.pants.graphql.goals.tailor.FixedPutativeTarget'>.
I think this is happening because of the code here: https://github.com/pantsbuild/pants/blob/41d4b82338cae0b54eb6e81d23dd0e9665fb2598/src/python/pants/core/goals/tailor.py#L586-L589
b

bitter-ability-32190

05/17/2022, 7:57 PM
post your code?
c

cold-soccer-63228

05/17/2022, 7:57 PM
Copy code
class FixedPutativeTarget(PutativeTarget):
    @classmethod
    def for_target_type(
        cls,
        target_type: type[Target],
        path: str,
        name: str | None,
        triggering_sources: Iterable[str],
        kwargs: Mapping[str, str | int | bool | tuple[str, ...]] | None = None,
        comments: Iterable[str] = tuple(),
    ) -> PutativeTarget:
        if name is None:
            name = os.path.basename(path)

        explicit_sources = (kwargs or {}).get("sources")
        if explicit_sources is not None and not isinstance(explicit_sources, tuple):
            raise TypeError("Explicit sources passed to PutativeTarget.for_target_type must be a Tuple[str].")

        default_sources = default_sources_for_target_type(target_type)
        if not (explicit_sources or triggering_sources or default_sources):
            raise AssertionError(
                f"A target of type {target_type.__name__} was proposed at "
                f"address {path}:{name} with explicit sources {', '.join(explicit_sources or triggering_sources)}, "
                "but this target type does not have a `sources` field."
            )
        owned_sources = explicit_sources or default_sources or tuple()
        return cls(
            path,
            name,
            target_type.alias,
            triggering_sources,
            owned_sources,
            addressable=True,  # "Real" targets are always addressable.
            kwargs=kwargs,
            comments=comments,
        )
Then, I substitute
FixedPutativeTarget
where I previously used
PutativeTarget
b

bitter-ability-32190

05/17/2022, 7:58 PM
šŸ™ˆ not what I meant when I said to copy the code lol
I meant where you were calling
for_target_type
instead just construct your
PutativeTarget(...)
Copy code
pts = [
        PutativeTarget(<put the right values here>)
        for dirname, filenames in group_by_dir(unowned_graphql_files).items()
    ]
c

cold-soccer-63228

05/17/2022, 7:59 PM
OH lol
šŸ˜‚ 1
šŸ˜›
Sorry, totally misunderstood
What should
type_alias
and
owned_sources
be?
b

bitter-ability-32190

05/17/2022, 8:01 PM
Just look at
for_target_type
and follow the code along
So
ResourcesGeneratorTarget.alias
and maybe
tuple()
?
c

cold-soccer-63228

05/17/2022, 8:07 PM
Hmm. So the following code doesn't seem to generate what I want.
Copy code
pts = [
    PutativeTarget(
        path=dirname,
        name="graphql",
        type_alias=ResourcesGeneratorTarget.alias,
        owned_sources=tuple(),
        triggering_sources=sorted(filenames),
    )
    for dirname, filenames in group_by_dir(unowned_graphql_files).items()
]
It generates the following.
Copy code
resource(
    name="graphql0",
)
Granted, this is generating something now, so it seems like a step in the right direction. I am wondering how I can get it to have something like follows.
Copy code
resource(
    name="graphql",
    sources=[ <the sources I passed in> ]
)
By the way, thanks for all of the generous help so far and holding my hand through this šŸ™
ā¤ļø 1
b

bitter-ability-32190

05/17/2022, 8:10 PM
Try setting
owned_sources
to something like
"*.graphql"
? Also super weird it is generating a
resource
and not a
resources
.
Hey I'm learning too!
c

cold-soccer-63228

05/17/2022, 8:16 PM
Hmm, still no luck. Tried setting
owned_sources
to: ā€¢
sorted(filenames)
ā€¢
("*.graphql",)
Also tried setting
addressable=True
. Perhaps I need to populate
kwargs
?...
b

bitter-ability-32190

05/17/2022, 8:18 PM
Clean out the changes
tailor
did to your BUILD file before trying a new permutation?
c

cold-soccer-63228

05/17/2022, 8:21 PM
Yup, I did. Seems like
PutativeTarget.for_target_type
is ultimately invoking the what I am currently invoking... Is there perhaps another rule I need to add besides
find_putative_targets
that I've defined? This tells me how to find the targets, but does it tell it also how to add the sources as I would expect?
b

bitter-ability-32190

05/17/2022, 8:21 PM
I doubt it
Yeah try
kwargs
šŸ¤·ā€ā™‚ļø
c

cold-soccer-63228

05/17/2022, 9:00 PM
Okay, so I don't know what's exactly going on, but I'm thinking something later in the callstack is maybe causing the
sources=[...]
not to be emitted. As an experiment, I tried invoking both
PutativeTarget.for_target_type(target_type=PythonSourcesGeneratorTarget, ...)
and
PutativeTarget(type_alias=PythonSourcesGeneratorTarget.alias)
and inspecting what was generated. If I don't pass in what the default sources of
PythonSourcesGeneratorTarget
into the
owned_sources
parameter of the
PutativeTarget
constructor, the sources aren't generated. But if I pass in the exact same default sources into the
owned_sources
parameter, they are generated... šŸ¤”
b

bitter-ability-32190

05/17/2022, 9:01 PM
Yeah thats a bummer šŸ˜•
Did
kwargs
work around that or nah?
c

cold-soccer-63228

05/17/2022, 9:02 PM
Nope, because
kwargs
is primarily used for
PutativeTarget.for_target_type
, and not the
PutativeTarget
constructor itself, really...
I just want to generate my dependencies for my GraphQL files šŸ˜­ šŸ˜­ šŸ˜­
b

bitter-ability-32190

05/17/2022, 9:03 PM
Feel proud, you're the first resources target generator šŸ†
c

cold-soccer-63228

05/17/2022, 9:03 PM
Oh, really? šŸ˜®
b

bitter-ability-32190

05/17/2022, 9:03 PM
Otherwise someone else would've stepped in this and pinged us, right?
h

hundreds-father-404

05/17/2022, 9:03 PM
sorry I was at lunch. what Pants version are you using?
šŸŒ® 1
šŸŸ 1
hehe Josh wrong country šŸ˜…
šŸŒÆ 2
c

cold-soccer-63228

05/17/2022, 9:11 PM
@hundreds-father-404, I'm on 2.11.0. Regarding the PR, I don't think that actually changes the boolean logic for the conditional?
Regarding the comment I made above, I noticed that this works to generate
python_sources
correctly. (I'm trying to generate
resources
, but am using
python_sources
generation as an experiment, since invoking
PutativeTarget.for_target_type
for
ResourcesGeneratorTarget
seems broken:
Copy code
PutativeTarget.for_target_type(
    target_type=PythonSourcesGeneratorTarget,
    path=dirname,
    name="graphql_schema",
    triggering_sources=sorted(filenames),
)
However, constructing
PutativeTarget
directly causes the
sources=[...]
to be missing in the generated files.
Copy code
PutativeTarget(
    path=dirname,
    name="graphql_schema",
    type_alias=PythonSourcesGeneratorTarget.alias,
    owned_sources=("*.graphql",),
    triggering_sources=sorted(filenames),
    addressable=True,
)
When inspecting the underlying
PutativeTarget
object, it seems that the only difference is the
owned_sources
attribute. @hundreds-father-404, I'm curious if you have any suspicions on why this causes
sources=[...]
to not generate?
h

hundreds-father-404

05/17/2022, 9:27 PM
im afk for next hour but will try to read this whole thread and look into it, sorry you're hitting this problem! if you have the time, a GitHub issue w/ your code would help a lot so I can reproduce Or even a comment here w/ your most up-to-date code
c

cold-soccer-63228

05/17/2022, 9:34 PM
Sure, here's the most up-to-date code. There are some potentially helpful code snippets as well in the most recent messages I've sent, which contain things I've tried.
Copy code
@dataclass(frozen=True)
class PutativeGraphQLTargetsRequest(PutativeTargetsRequest):
    pass


@rule(level=LogLevel.DEBUG, desc="Determine candidate GraphQL resources to create")
async def find_putative_targets(
    req: PutativeGraphQLTargetsRequest, all_owned_sources: AllOwnedSources
) -> PutativeTargets:
    all_graphql_files = await Get(Paths, PathGlobs, req.search_paths.path_globs("*.graphql"))
    unowned_graphql_files = set(all_graphql_files.files) - set(all_owned_sources)
    return PutativeTargets(
        [
            PutativeTarget(
                path=dirname,
                name="graphql_schema",
                type_alias=ResourcesGeneratorTarget.alias,
                owned_sources=("*.graphql",),
                triggering_sources=sorted(filenames),
                addressable=True,
            )
            for dirname, filenames in group_by_dir(unowned_graphql_files).items()
        ]
    )
Thanks for the help!
Okay, so it seems like if I provide
owned_sources=tuple(...)
and
kwargs={"sources": tuple(...)}
, then it works. But this feels like a bug?
šŸ‘€ 1
šŸ™Œ 1
I ended up filing an issue here: https://github.com/pantsbuild/pants/issues/15527, since I don't know if doubt the boolean fixes things automatically, given the weird issue I've seen with
kwargs
.
āœ… 1
3 Views