Hi, folks. I've spent some time reading up on the basic concepts of pants and plugins, and am now trying to put into words a description of what I want my plugin to do. I find myself struggling to write such a description. I'm kind of starting to doubt if a pants plugin can even do what I want to do. So, here's what I want to do, described in plain terms: 1. Build some docker images (one image for every service in a microservice application)* 2. Use the names of the services and the image tags of the docker images to modify a YAML file *This is already done with pants. Somebody else has set this up and it works, so I'm not too concerned with this part. The YAML file should contain contain the service names and image tags. Something like this:

tags:
  - core-service-bc0b936
  - document-service-ccb4811
  - measurement-service-f8b293a

I have a number of questions: • Can pants write to a file like this? I would assume that it can since writing files is kind of what a build system does. However, so far it seems to me that pants plugins should be functional, that is, without side effects. Their rules should take an input and return an output. So I guess a pants plugin can't do something like this? How would you then perform an operation like this? Do I actually want to write a goal rule, not a plugin rule? • How do I figure out the input types I need? I guess I want to take in some kind of type that could be found in the docker pants plugin, but I don't know how I can figure out what the types mean. Perusing the source code to find the types I want would probably involve a lot of studying and thinking to reverse engineer the entire docker plugin, which feels very heavy. Any help is greatly appreciated. I hope I don't come across too critical, I'm just kinda confused and feeling a bit hopeless. Pants seems cool and I've learned quite a bit about many of the concepts, but it still feels like I am miles away from doing anything useful. All I have after 10 hours of research is a hello world and a lot of notes to myself that may or may not make sense🥲

Personally, I'd just write a script that calls pants and then generates the YAML file, instead of trying to get it to work inside of Pants. Pants has the capabilities (sort-of), but it can be tricky and likely won't have the best UX IMO.

Oh man 🙈

Hi 👋 You can do anything with Pants, as it’s executin generic Python in the end. For side-effecting stuff (especially files) there’s some caveats and new idioms to learn however. What you want to learn if you’re digging further into Pants is the concept about Digests. As Pants need to capture the state of files, all I/O goes through the engine in terms for reading and writing these file Digests. The relevant page in the docs describing this is https://www.pantsbuild.org/docs/rules-api-file-system As @bitter-ability-32190 said, it may be more productive to learn Pants by using it first and once you’re comfortable with how it works as a user then swim over to the deep end of the pool and start using the Plugin API. 🙂 Do what makes most sense for you 😉

Surely you are allowed to build any file you could dream of if you write a plugin for it?

You probably CAN, but SHOULD you? 🙂 I'm wondering (like, genuinely wondering) if maybe you could run your scripts using an

experimental_shell_command

? Using the rules that finish the docker builds, and then run a script that will do what you want? That way I think it should still be able to tell when your deps are dirty, and re-run as needed. ... I think.

Hi! You correctly intuited that only a

@goal_rule

can side-effect into files in your repo. So typically you'd write a custom goal that requests the metadata of the built Docker images (which will then cause them to be built if necessary) and writes that out into the yaml file. If the goal rule needs to call other rules, they can pass file contents back to it in the form of a Digest.

Oh, but one important question: In step 2, are the names of the services and the image tags of the docker images available outside of Pants, e.g., by inspecting the image in Docker directly? Or is this information only known to Pants as a byproduct of creating the images?

I don't know where the tags are available. The idea is to use

--changed-since

to only build changed code. After that I have no clue how to see which images were built.

For 2.16 there’ll be some metadata about the images built in

dist/

See: https://github.com/pantsbuild/pants/commit/23cc363a23ba5f0b1f934c2a6e2f498cf30ab0b6