topic Re: Alerts for Failed Queries in Databricks in Data Engineering

Alerts for Failed Queries in Databricks

AcrobaticMonkey — Tue, 03 Dec 2024 10:32:29 GMT

How can we set up automated alerts to notify us when queries executed by a specific service principal fail in Databricks?

Re: Alerts for Failed Queries in Databricks

Alberto_Umana — Tue, 03 Dec 2024 13:45:00 GMT

Hi @AcrobaticMonkey,

How are you triggering the queries using a service principal, is it through a workflow job? If so then you can use Job notifications.

I would need more details but one possible solution is to implement Alerts:

https://docs.databricks.com/en/sql/user/alerts/index.html

Re: Alerts for Failed Queries in Databricks

AcrobaticMonkey — Fri, 06 Dec 2024 05:27:27 GMT

@Alberto_UmanaOur service principal uses the SQL Statement API to execute queries. We want to receive notifications for each query failure. While SQL Alerts are an option, they do not provide immediate responses. Is there a better solution to achieve near-real-time failure notifications?