I'm starting to think about ways to possibly extend pants for our team using custom plugins to wrap certain actions in our build. An example would be something somewhat similar to deploying a lambda function, instead with dagster - build a container that has a pex file, upload to ECR, hit the Dagster API to notify of the new image tag so they can pull it. I'm considering wrapping this whole thing up into a single target, but one thing I've noticed is that I don't think I've seen existing targets that hit any kind of external service. They usually seem to build some kind of artifact and then leave it up to downstream processes to actually do something with them (although I guess that's not really true in the case of publishing, but for some arbitrary reason that feels different?). Are there any technical or philosophical reasons why I would want to exclude the step where I notify the Dagster API of the new container from being governed by a pants target?

I wonder if @abundant-autumn-67998 has any thoughts on this.

I see the uploading to ECR as a

publish

step and notifying the dagster API as a

deploy

step. It could very well be represented as a single target, and depending on the goals you run pants with do more or less of it at any given time.

The docker plugin is a good exemplar of something that interacts with external state. You do have to be careful when doing so. For example, if you can structure your operations to be idempotent that will save you a world of pain. And you may need to mark certain processes as uncacheable, if they must run to sync state

yeah that makes total sense, I had been thinking it might take a little more forethought, you make a great point about trying to keep things idempotent if at all possible. seems like something worth continuing to explore then - thank y'all for your input!

I agree deploying to dagster sounds like a "deploy" step. I'm not super familiar with pants but looks like there is alpha support for helm deployments. Implementing this as something analogous for dagster, eg a

dagster_deployment

target might work.

fwiw, the helm support has been around for a good while. We're just extremely cautious about moving things out of

experimental

. 😉