Ergonomic, hermetic, user-friendly software build system for Python, Go, Java, Kotlin, Protobuf, Scala, Shell. Pants lets you fearlessly scale up your codebase!

Pants

Hi all
I'm planning to use Airflow with CeleryExecutor. I have existing services built with Pants, each packaged as a pex file. My question is: can I run a pex file as a task in a Celery worker, or should I extract the code and run it directly as Python in the Celery worker? What's the recommended approach for running tasks in Airflow?

I don’t know the best way but I know you can take a pex and make a virtual environment out of it and then use that as your runtime environment for your tasks

You can also override a pex entry point with environment variables, ie PEX_MODULE or PEX_SCRIPT

I strongly recommend reading the pex docs. I didn’t do that originally and regret it in hindsight. It’s relatively short and full of great information. 