https://pantsbuild.org/ logo
#general
Title
# general
p

plain-carpet-73994

04/28/2021, 4:24 PM
Hi. Just getting started with Pants and need to write some custom rules. I'm a little confused about how to chain rules together. For example, suppose my rule consume target
my_target
and generates some new files. As I understand it, I can return a custom`Goal` subclass that contains some information about those files but that does not generate new targets. Now suppose I want to make sure my generated code is safe via
bandit
. Since I don't have a new target and
bandit
's
@rule
doesn't take a
MyCustomGoal
how do I tell
bandit
that it depends on (or should check) these custom files?
👋 2
h

hundreds-father-404

04/28/2021, 5:07 PM
Hi Oliver, welcome! Happy to help. Taking a step back, it sounds like your plugin is trying to generate Python from some other format, and then you want Bandit to run on that generated code. Is that right?
p

plain-carpet-73994

04/28/2021, 5:10 PM
Yea, that was the quesiton. It's actually a bit more complicated than that but I figured the easy way to ask the question would be to pretend that was the case. If I could do python gen -> bandit run I think I could solve my actual issue.
i guess the real issue is that I'm a bit confused about how to chain together the output generated by one rule and specify that output should be the input to another target or rule. In particular, rules don't output targets but they do generate files so I don't quite understand how to refer to those generated files.
h

hundreds-father-404

04/28/2021, 5:13 PM
Are you specifically trying to run Bandit, which is something already written? Or are you in total control of the whole pipeline and trying to run a tool that you control?
p

plain-carpet-73994

04/28/2021, 5:14 PM
I am in control of the total pipeline. That was just an example. But conceptually I understand that if a rule,
A
, that returns a
Foo
you can have another rule,
B
, that requests a
Foo
as input so when that goal
B
is requested Pants knows it need to run the rule
A
first. What I don't understand is the more general case like
bandit
where it can't know what generated files might exist beforehand. If I generate a file how can I "talk about it" in other targets and rules?
Sorry if I'm not making much sense...
h

hundreds-father-404

04/28/2021, 5:16 PM
rules don't output targets but they do generate files
To clarify, rules can output whatever they want (as long as it's a single type with a hash and eq implementation). Rules can output targets, `Digest`s representing files, some custom type, etc
specify that output should be the input to another target or rule
You either put that output type as an argument to your
@rule
function, or use
await Get(OutputType, OutputTypeRequest)
in the calling rule. Which format you use depends on if Pants needs some input in order to compute
OutputType
vs. if it can be computed without extra input from the callsite
p

plain-carpet-73994

04/28/2021, 5:18 PM
Hmmm... I think I understand what you're saying but I'm still missing something. For example, suppose I have a
blob.foo
file in
some/dir
and my custom rule generates a
blob.py
from that. If I want something else to depend on
blob.py
how do I refer to it? Can I put a
some/dir/blob.py
in the
dependencies
section of something like a
package
or
bandit
rule? I suspect not 'cause I don't think Pants wold know that file exits unless I specifically
pants my_rule some/dir/blob.foo
, right?
h

hundreds-father-404

04/28/2021, 5:19 PM
What I don't understand is the more general case like bandit where it can't know what generated files might exist beforehand.
The caller needs to know about the output type so that it can request the output type There is a way to make that more generic through a thing called unions, which are polymorphism for the engine. This is how all of Pants's plugin hooks work - we have no idea what plugins users might right, but unions allow us to handle that. Although, most the time plugins don't need to set up their own unions because you are usually in total control of your pipeline https://www.pantsbuild.org/docs/rules-api-unions
p

plain-carpet-73994

04/28/2021, 5:19 PM
I think the union rules is maybe the only section of the API docs I haven't read 🙂 I can give that a read.
👍 1
h

hundreds-father-404

04/28/2021, 5:21 PM
With codegen, is that only an example or you do want to use codegen in your plugin? If so, there is already prior art for how to do that, including a plugin hook https://www.pantsbuild.org/docs/rules-api-and-target-api#enabling-codegen https://www.pantsbuild.org/docs/plugins-codegen
p

plain-carpet-73994

04/28/2021, 5:21 PM
Rules can output targets
They can? I thought the pattern was:
Copy code
class MyGoalSystem(GoalSubsystem):
    # ...

class MyGoal(Goal):
    # ...

@goal_rule
fun my_rule(...) -> MyGoal:
   # ....
that is, I thought
MyGoal
had to be a subclass of
Goal
and my
@goal_rule
had to return that.
h

hundreds-father-404

04/28/2021, 5:23 PM
Ah, so that is only if you want to register a new goal, like a new verb that people can run. A lot of plugins never add new goals and ~90% of rules are not goal rules With goal rules, though, indeed they must return a subclass of
Goal
p

plain-carpet-73994

04/28/2021, 5:23 PM
got it!
❤️ 1
I still don't have it all in my head but the pointers to the docs were super helpful! I'll do some reading and get back to you if I'm still confused. Thanks a ton for your help!
❤️ 1
h

hundreds-father-404

04/28/2021, 5:25 PM
You're welcome! There is certainly a learning curve, and also that is only the first version of those docs - we know we want to improve them, so your feedback is helpful
p

plain-carpet-73994

04/28/2021, 5:35 PM
I'm coming from Bazel so have some familiarity with the concepts. Finding your docs to be better than theirs.
❤️ 1
👌 1
3 Views