https://pantsbuild.org/ logo
c

cold-soccer-63228

05/18/2022, 2:37 PM
(This is a follow-up to a previous super-long thread, but I figured I'd start a new thread since this is a slightly (?) different question.) Hello, I've been able to generate the following resources in various
path/to/service/graphql/BUILD
files, where
*.graphql
files are siblings in the directories by subclassing
PutativeTargetsRequest
.
Copy code
resources(
    name="graphql_schema",
    sources=[
        "*.graphql",
    ],
)
I'm interested in making it so that Python sources that invoke
ariadne.load_schema_by_path
have dependencies on all GraphQL resources in my codebase, e.g. the targets specified in each
path/to/service/graphql/BUILD
file. (This is necessary because the
ariadne.load_schema_by_path
function opens GraphQL file paths.) I think the
BUILD
file I want to generate should look something like the following:
Copy code
python_sources(
    dependencies=[
        "path/to/service_a/graphql:graphql_schema",
        "path/to/service_b/graphql:graphql_schema",
        "path/to/service_c/graphql:graphql_schema",
        ...
    ]
)
I see that infer_python_dependencies_via_source is a rule that already exists in the Python backend. I was wondering if I could leverage this. To solve my problem, would it be valid to create an additional rule, e.g.
infer_python_graphql_dependencies
that awaits
Get(InferredDependencies, ...)
, and then looks through the dependencies, and adds dependencies on all
"path/to/service/graphql:graphql_schema"
targets whenever an
ariadne
dependency is found? So far, I have the following (not working) code that I'm trying to play with.
Copy code
class InferPythonGraphQLResourceDependenciesRequest(InferDependenciesRequest):
    infer_from = PythonSourceField


@rule(desc="Inferring Python GraphQL dependencies by analyzing source")
async def infer_python_graphql_dependencies(
    request: InferPythonGraphQLResourceDependenciesRequest,
    python_infer_subsystem: PythonInferSubsystem,
    python_setup: PythonSetup,
) -> InferredDependencies:
    if not python_infer_subsystem.imports and not python_infer_subsystem.assets:
        return InferredDependencies([])

    _wrapped_tgt = await Get(WrappedTarget, Address, request.sources_field.address)
    tgt = _wrapped_tgt.target
    return await Get(InferredDependencies, DependenciesRequest(tgt[Dependencies]))


def rules() -> Iterable[Rule]:
    return [
        infer_python_graphql_dependencies,
        UnionRule(InferDependenciesRequest, InferPythonGraphQLResourceDependenciesRequest),
    ]
The issue with this code are errors that look like the following:
Copy code
No installed rules return the type InferConftestDependencies
No installed rules return the type InferInitDependencies
No installed rules return the type InferPythonGraphQLResourceDependenciesRequest
Do I perhaps I need to re-register rules the
rules()
function defined in my dependency inference declarations?
b

bitter-ability-32190

05/18/2022, 2:59 PM
Do I perhaps I need to re-register rules the
rules()
function defined in my dependency inference declarations?
Make sure your
register
has a
rules()
which returns these rules
1
I'm interested in making it so that Python sources that invoke
ariadne.load_schema_by_path
have dependencies on all GraphQL resources in my codebase
Once you get the rule running, what you want should be easy. • Take the
python_source
you're given, ask for the contents, parse it for the
ast.Call
you're looking for. • Have your Rule also take in AllAssetTargetsByPath , and filter to only those with
.graphql
paths • If the python source has the right call, return an object with the resource targets for graphQL resources
c

cold-soccer-63228

05/18/2022, 3:13 PM
Have your Rule also take in AllAssetTargetsByPath , and filter to only those with
.graphql
paths
Would this be an additional parameter in the the function signature? Or perhaps do I make my request subclass both
InferDependenciesRequest
and
AllAssetTargetsRequest
? Or perhaps something else?
b

bitter-ability-32190

05/18/2022, 3:15 PM
In this case, make it an additional argument to your rule. Normally you need to
await Get(...)
to have the engine take your request and run the right code (or grab form the cache) the result. In a few cases though there ARE no inputs (or more accurately the request object is just dataless).
AllAssetTargetsByPath
is one of those. In that case you can just declare it as an argument and the engine will provide it
c

cold-soccer-63228

05/18/2022, 3:23 PM
Hmm, if I provide it as an additional parameter, then I get an error.
Copy code
No installed rules return the type AllAssetTargetsRequest, and it was not provided by potential callers of @rule(pants.core.target_types:326:find_all_assets(AllTargets, AllAssetTargetsRequest) -> AllAssetTargets).
    If that type should be computed by a rule, ensure that that rule is installed.
    If it should be provided by a caller, ensure that it is included in any relevant Query or Get.
It seems like doing the
await Get(...)
worked though? But I'm not sure if my dependency inference is getting triggered. I'm using the following code the test.
Copy code
async def infer_python_graphql_dependencies(request: ...) -> InferredDependencies:
    all_asset_targets = await Get(AllAssetTargets, AllAssetTargetsRequest())
    assets_by_path = await Get(AllAssetTargetsByPath, AllAssetTargets, all_asset_targets)
 
    print("HUGH-TEST")
    print(request.sources_field)

    return InferredDependencies([])
I'm noticing that
"HUGH-TEST"
never gets printed?
b

bitter-ability-32190

05/18/2022, 3:23 PM
Rules are run asynchronously, and therefore cannot print
c

cold-soccer-63228

05/18/2022, 3:23 PM
Ahhhh
b

bitter-ability-32190

05/18/2022, 3:23 PM
Try instantiating a logger (
logging.getLogger(__name__)
)
and then logging
.error(...)
thats what I do 😛
Copy code
No installed rules return the type AllAssetTargetsRequest
Make sure your argument type is
AllAssetTargetsByPath
. Seems like your type was the
XRequest
type?
c

cold-soccer-63228

05/18/2022, 3:40 PM
Make sure your argument type is
AllAssetTargetsByPath
. Seems like your type was the
XRequest
type?
Yup, this function signature triggers that above error.
Copy code
async def infer_python_graphql_dependencies(
    request: InferPythonGraphQLResourceDependenciesRequest, _: AllAssetTargetsByPath
) -> InferredDependencies:
    ...
b

bitter-ability-32190

05/18/2022, 3:41 PM
Aww guess I misunderstand the engine 😕
😭 1
I think if the engine had a rule which took nothing and returned
AllAssetTargetsRequest
it might work? But I'm not sure 😅 . Making the
await Get
isn't wrong so 🤷‍♂️
c

cold-soccer-63228

05/18/2022, 4:37 PM
Hmm. If I want to make it so that I inject all of the GraphQL dependencies whenever I see an
ariadne
import, can I piggy-back off of the Python dependency inference script somehow?
b

bitter-ability-32190

05/18/2022, 4:37 PM
I don't see why not? You might even just request the parsed info
Use
ParsePythonDependenciesRequest
? Suggest using the same values as in
src/python/pants/backend/python/dependency_inference/rules.py
so you get cached results
You might actually try using a
InferPythonImportDependencies
instance to get the
InferredDependencies
for your source 🙈
Up to you
c

cold-soccer-63228

05/18/2022, 4:59 PM
Just to clarify, adding a dependency inference rule wouldn't change the
BUILD
file, right? But it would implicitly make it so that targets depend on some other targets?
b

bitter-ability-32190

05/18/2022, 5:00 PM
Yup!
You can verify it works by doing
./pants dependencies path/to/file.py
c

cold-soccer-63228

05/18/2022, 5:05 PM
Oh, very cool. Thanks! I think that the
target_name
of my
ariadne
address is just showing up as
"requirements"
since it's a third party dependency. Is there a field that I can check to see that it's something like
"requirements#ariadne"
?
Okay, it looks like it's
Address.spec
Okay, it seems like
./pants dependencies ...
is showing the GraphQL resources now properly inferred, whenever we import
ariadne
🙂 I know we briefly touch upon this earlier with @hundreds-father-404, but do you think it'd be worthwhile to put up a dummy merge request with the code that I wrote? This may end up being useful for other engineers that use GraphQL with the
ariadne
package. It's very barebones, but it could be a starting point if we ever wanted to build a
ariadne
or
graphql
backend
1
🙌 1
🎉 1
To pick your brains a bit—what do you think such backends would look like? I was thinking a GraphQL backend would create resources for GraphQL files and also provide a linter/formatter. There could be a separate ariadne plugin (not sure if this would be in the Pants or perhaps Ariadne repo) that then infers dependencies whenever the ariadne library is used. What are all of your thoughts?
b

bitter-ability-32190

05/19/2022, 8:40 AM
If you plan on adding a formatter/linter, then the targets get upgraded from resources to dedicated targets (like we very originally talked about). I'm not sure if we'd be able to do any dep inference outside of normal asset inference. As for the plugin, it seems a bit niche but I'm on board for more support for how third party libraries do things. It'd take convincing though
1
FWIW the reason I say that, too. Is that pants currently has plugins for tools and languages. We haven't yet seen a plugin for data files (although like I said the fmt/lint aspect makes sense), or for a single package.
👍 1
c

cold-soccer-63228

05/19/2022, 2:26 PM
Mmmm I see that makes sense. Perhaps it could make sense to publish it as a stand alone plugin solely for Ariadne in a separate repository? I was thinking: “anyone using Ariadne will end up having to rewrite something like I wrote”…
1
h

hundreds-father-404

05/19/2022, 2:49 PM
For sure! That would be great. We intentionally haven't invested a ton yet into publishing infrastructure of plugins because the Plugin API is not stable. But it is possible to publish to PyPI A good first step is simply to open a repository so that people can copy and paste your code, e.g. https://github.com/sureshjoshi/pants-plugins
👍 1
c

cold-soccer-63228

05/19/2022, 6:22 PM
Cool, here it is: https://github.com/hughhan1/pants-plugins It's under the
ariadne
directory. Please lemme know if y'all have any suggestions or anything. I'm thinking of adding tests to this repository, and then publishing this barebones plugin as a PyPI package, since the
ariadne
plugin name conflicts with our
ariadne
third-party package name in our production codebase 😕
❤️ 1