4 weeks ago - last edited 4 weeks ago
I'm looking for SQL linters in two different contexts within Databricks GUI:
SQL queries in Databricks SQL Editor (GUI) - Is there a built-in linter for writing SQL queries in the Databricks SQL workspace?
%sql magic cells in Databricks notebooks (GUI) - Is there a linter that checks SQL syntax/quality within %sql cells in notebooks accessed through the Databricks GUI?
CLI usage - If SQL linting capabilities exist for either of the above, is there a way to use them via Databricks CLI (for CI/CD pipelines, for example)?
I've already found that Databricks uses gethue/sql-formatter for SQL formatting in both contexts, but I haven't been able to find any information about corresponding linters. I've searched through the documentation and community forums without success.
Does Databricks provide built-in SQL linting capabilities in either of these environments? I would prefer to use an official Databricks-recommended approach if one exists. If not, are there any officially endorsed or commonly recommended third-party tools that integrate well with Databricks for SQL linting?
Any guidance would be appreciated!
4 weeks ago
in databricks notebook is possible to connect own linters but I don't think it will work in SQL editor.
4 weeks ago - last edited 4 weeks ago
in databricks notebook is possible to connect own linters
Thank you for your response. I can confirm that depending on the Databricks Runtime version, linters such as Ruff, Pylint, and Flake8 are available for use(ref). However, I haven't been able to find any linter specifically for `%sql` cells in Databricks notebooks (those with the `%sql` magic command) within the workspace. Should I understand that SQL linting is not currently supported for these cells?
I don't think it will work in SQL editor.
Thank you for clarifying. So far, I haven't been able to find any SQL linter support either.
3 weeks ago
You can check open source sqlfluff. But I guess this option will be suited for CLI usage. (and use within some CI/CD process)
3 weeks ago
Thank you. I think sqlfluff is perfect as it's effective for .sql files and supports the Databricks dialect. The issue is with %sql cells in .ipynb files. To handle this, there are tools like sqlparse and nbqa, but there are concerns about whether they support the Databricks dialect and the fact that I'd prefer not to rely on unofficial tools. Also, since I don't like having a mix of tools, I was hoping to find something that could be used uniformly for both.
Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!
Sign Up Now