https://pantsbuild.org/ logo
Join Slack
Powered by
Pants monorepo for a company is not as attractive ...
# general
b
Pants monorepo for a company is not as attractive as Bazel monorepo. Python is the first class language of Pants. Modern Data/ML Stack relies heavily on Python. Let Data/ML dance with Pants!
👯 6
h
We do have a lot of use in the data science space, and those use-cases are one of the reasons we focused on Python initially, and then Scala next.
I’d argue that Pants is a better choice for many other areas as well, but Data Science/ML is where we should really shine.
However, not many of the core Pants maintainers have much personal data science experience as a user, so we welcome suggestions, designs etc!
b
I think the most important thing for Pants on Data/ML is adding SQL support now!
h
What kind of SQL support would be useful?
b
Just like the shell language support, format and lint is good enough for the sql language. Using SQL Lint tool like: https://docs.sqlfluff.com/en/stable/
For Data Engineering, we care about Redshift SQL/Spark SQL/Google BigQuery/… It is covered in sqlfluff: https://docs.sqlfluff.com/en/stable/dialects.html
And for backend purpose, an integration with flyway would be a great next step.
Things like dependency inference are called sql lineage: https://github.com/reata/sqllineage This is the third step.
dbt is the super star in data stack using SQL and Python.
h
That’s all great info
Would you mind opening a ticket for this, with all those links and details?
👌 2
Then we can track it
5 Views
Open in Slack
PreviousNext
Powered by