08/17/2022, 10:07 AM
Pants monorepo for a company is not as attractive as Bazel monorepo. Python is the first class language of Pants. Modern Data/ML Stack relies heavily on Python. Let Data/ML dance with Pants!
08/17/2022, 2:56 PM
We do have a lot of use in the data science space, and those use-cases are one of the reasons we focused on Python initially, and then Scala next.
I’d argue that Pants is a better choice for many other areas as well, but Data Science/ML is where we should really shine.
However, not many of the core Pants maintainers have much personal data science experience as a user, so we welcome suggestions, designs etc!
08/18/2022, 1:11 AM
I think the most important thing for Pants on Data/ML is adding SQL support now!
08/18/2022, 3:48 AM
What kind of SQL support would be useful?
08/18/2022, 3:52 AM
Just like the shell language support, format and lint is good enough for the sql language. Using SQL Lint tool like:
For Data Engineering, we care about Redshift SQL/Spark SQL/Google BigQuery/… It is covered in sqlfluff:
And for backend purpose, an integration with flyway would be a great next step.
Things like dependency inference are called sql lineage:
This is the third step.
dbt is the super star in data stack using SQL and Python.
08/18/2022, 4:21 AM
That’s all great info
Would you mind opening a ticket for this, with all those links and details?
Then we can track it