https://pantsbuild.org/ logo
#general
Title
# general
h

high-energy-55500

07/26/2022, 2:42 PM
hey guys, I’m setting up airflow and was wondering if there are any best practices or recommendations in getting airflow set up in our monorepo. specifically: 1. we’re using poetry to track all our common dependencies, but since airflow pins its dependencies and doesn’t support poetry, we’re not able to simply do a
poetry add apache-airflow
since it leads to an unsatisfiable list of dependency constraints 2. because of this, airflow requires you provide a
requirements.txt
with a list of 3rd-party libraries used in your DAGs. does pants have a way to generate a requirements file from a
python_sources
target? am i thinking about this the wrong way? 3. haven’t started working on adding support for 1st-party code that’s imported into DAGs but I have a feeling this is not going to be very straightforward. any recommendations here?
3 Views