10/21/2021, 5:37 PM
So someone at my company was looking to solve a large ongoing problem (how to reliably build cloud images) and came up with the following solution:
This whole thing fits very naturally as a graph of dependencies. We could model it this way and execute nodes in the graph as their dependencies are satisfied, and if nodes fail we can retry N times before marking them completely failed.
I built a little test executor for this using networkx and it looked like it could work pretty well. It basically worked like:
- build graph, eg base image depends on container, provider image depends on base image, provider upload depends on provider image etc.
Then, to execute:
- gather all stages that are runnable (dependencies are satisfied)
- submit to threadpool to run
- sleep
- walk the graph again and find any stages that are now runnable and submit to threadpool to run
- sleep
- continue until we can't progress any more or everything is done
If only he knew there was tooling that already basically did this, but in a little more refined way. 😀 Looks like I've got another person I can make a good case for Pants to.
👖 5
💯 2


10/21/2021, 5:43 PM
Now taking “pantsify” to a new level, not only for projects.. 😆