The second part is a bit more complicated. Each of the applications of the monorepo is run using Airflow, and for this each application dynamically generates a set of tasks that are then put together by the tooling. The process currently is as follows: 1. We call the

build-dags

command in our custom tool 2. For each of the libraries inside the

app

folder, the tool loads

library.dag

and executes a

generate_tasks

function, which returns a list of airflow tasks and their dependencies. 3. Once all tasks have been collected, the tool figures out the DAGs (you can have cross-app DAGs) and writes an output YAML file that is deployed from the CI in airflow. How to achieve that with

pants

? My idea is to break this into two steps: 1. On

./pants package

, not only the

pex

are generated but also YAML files with the tasks for each app/library. This would mean creating a new target

airflow_tasks

that would load the

library.dag

, run the

generate_tasks

function and dump the result in a per-app YAML. 2. Create a new goal

./pants build-dags

(or maybe it could be done with

package

if

pants

supports “ordered” builds) that has the

airflow_tasks

as inputs and then puts together all the YAMLs and builds the DAGs. I guess this is how I would do it, but looking at the plugins docs it all seems a bit daunting. Any example plugins I can look at to understand how to do this myself?