So someone at my company was looking to solve a large ongoing problem (how to reliably build cloud images) and came up with the following solution:

This whole thing fits very naturally as a graph of dependencies. We could model it this way and execute nodes in the graph as their dependencies are satisfied, and if nodes fail we can retry N times before marking them completely failed.

I built a little test executor for this using networkx and it looked like it could work pretty well. It basically worked like:

- build graph, eg base image depends on container, provider image depends on base image, provider upload depends on provider image etc.

Then, to execute:

- gather all stages that are runnable (dependencies are satisfied)

- submit to threadpool to run

- sleep

- walk the graph again and find any stages that are now runnable and submit to threadpool to run

- sleep

- continue until we can't progress any more or everything is done

If only he knew there was tooling that already basically did this, but in a little more refined way. 😀 Looks like I've got another person I can make a good case for Pants to.

Now taking “pantsify” to a new level, not only for projects.. 😆