Hi everyone. I work at Ledger (the hardware wallet company for crypto assets) and I am a data engineer. I introduced Pants few weeks ago in the data team, to handle all our data projets in a monorepo using Pants. At Ledger, we have to handle a lot of data from multiple sources (APIs, blockchains, etc.), so my idea was to have one Python/Scala project by data source, to package each project in a PEX file/Docker image, to publish the Docker images in Amazon ECR, and to schedule all pipelines using Apache Airflow (which will up a container in AWS Fargate for each data task). Currently I developed that in a sandbox environment and I mostly rely on Pants commands to do these steps.
05/11/2022, 2:19 PM
Cool! Thanks for sharing that!
How has the Pants integration been going?