cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Creating Databricks Alerts/Notifications without using the UI

lurban
New Contributor

I am trying to work on a Databricks notebook that can dynamically create alerts using the Databricks API (Documentation linked here: https://docs.databricks.com/api/azure/workspace/alerts/list). I can successfully create an alert, however what is missing is the ability to assign notifications. There is nothing in the documentation that indicates it as an option to configure notification settings or to enable these. The closest thing is to be able to set up a custom notification subject/body, but this would not be useful if a notification cannot be configured.

Am I missing something in being confused why this capability is not there and is the only way truly to use the UI to configure notification destinations? 

Has anyone identified a successful workaround to be able to fully set up an alert complete with notifications configured without any use of the UI? 

Thanks in advance!

 

6 REPLIES 6

Jerry01
New Contributor III

I am also running into a similar issue. Using databricks alert API, how to give refresh schedule, status, Email notifications, etc. I tried giving status as triggered but the alert is always getting created with unknown

SaugatMukherjee
New Contributor III

The way I hopefully figured this out (am going to try it out in a couple of days), was first doing it manually, getting an alert out to a teams channel (configured as a destination) , using the default template. When it actually wrote on teams, I got a link to a job (which apparently isnโ€™t visible on the workflows page). Looking at the yaml of the job, it seems whenever a schedule is added to an alert, it creates a workflow with a โ€œsql_taskโ€. 
Now, we have already been writing to teams channel, for failed jobs using email notifications and using the mail id of the channel (yes, there is an email link for a teams channel).

So, I would approach this as:

1. Create a saved query (through api, you can use terraform too).

2. Create an alert for the query. This only creates an alert with no schedule.

3. Create a workflow/job through api with a sql task pointing to an alertid and using email_notifications for the job (email notification can be used for both teams and to normal mailing groups).

I hope this should work, but I have to try it out and confirm in a few days.

This should however have to be simpler.

I can now confirm that this works.

1. I created a saved query using databricks rest api.

2. Created an alert using databricks rest api. This does not include the schedule.

3. Created a job with a task of sql_task type with the alert id assigned to the alert created in step 2, again using the rest api.

While creating the job, you can also set email_notifications - that is also part of the payload if you look at the rest api definition for jobs.

subbaram
New Contributor II

Hi Saugat,

Is there a way that i can get the owner of a schema/table dynamically and send out a notification to him using Rest API.

Hi Subbaram,

You should post a new question for this. I will then try to answer.

Hi Saugat,

Just wondering how exactly you got the job url when you manually created the alert in the UI? I am calling to the jobs api and can't find any instances of the alert-scheduled sql query running.

Thanks!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group