cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Are there SQL linters for Databricks GUI queries and notebook %sql cells?

Lenoq
New Contributor

 

I'm looking for SQL linters in two different contexts within Databricks GUI:

  1. SQL queries in Databricks SQL Editor (GUI) - Is there a built-in linter for writing SQL queries in the Databricks SQL workspace?

  2. %sql magic cells in Databricks notebooks (GUI) - Is there a linter that checks SQL syntax/quality within %sql cells in notebooks accessed through the Databricks GUI?

  3. CLI usage - If SQL linting capabilities exist for either of the above, is there a way to use them via Databricks CLI (for CI/CD pipelines, for example)?

I've already found that Databricks uses gethue/sql-formatter for SQL formatting in both contexts, but I haven't been able to find any information about corresponding linters. I've searched through the documentation and community forums without success.

Does Databricks provide built-in SQL linting capabilities in either of these environments? I would prefer to use an official Databricks-recommended approach if one exists. If not, are there any officially endorsed or commonly recommended third-party tools that integrate well with Databricks for SQL linting?

Any guidance would be appreciated!

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III

in databricks notebook is possible to connect own linters but I don't think it will work in SQL editor.

in databricks notebook is possible to connect own linters


Thank you for your response. I can confirm that depending on the Databricks Runtime version, linters such as Ruff, Pylint, and Flake8 are available for use(ref). However, I haven't been able to find any linter specifically for `%sql` cells in Databricks notebooks (those with the `%sql` magic command) within the workspace. Should I understand that SQL linting is not currently supported for these cells?

 

I don't think it will work in SQL editor.


Thank you for clarifying. So far, I haven't been able to find any SQL linter support either.

szymon_dybczak
Esteemed Contributor III

You can check open source sqlfluff.  But I guess this option will be suited for CLI usage. (and use within some CI/CD process)

szymon_dybczak_1-1761506465088.png

 

sqlfluff/sqlfluff: A modular SQL linter and auto-formatter with support for multiple dialects and te...

Thank you. I think sqlfluff is perfect as it's effective for .sql files and supports the Databricks dialect. The issue is with %sql cells in .ipynb files. To handle this, there are tools like sqlparse and nbqa, but there are concerns about whether they support the Databricks dialect and the fact that I'd prefer not to rely on unofficial tools. Also, since I don't like having a mix of tools, I was hoping to find something that could be used uniformly for both.