topic Re: Creating Databricks Alerts/Notifications without using the UI in Data Engineering

Creating Databricks Alerts/Notifications without using the UI

lurban — Wed, 02 Aug 2023 17:02:38 GMT

I am trying to work on a Databricks notebook that can dynamically create alerts using the Databricks API (Documentation linked here: https://docs.databricks.com/api/azure/workspace/alerts/list). I can successfully create an alert, however what is missing is the ability to assign notifications. There is nothing in the documentation that indicates it as an option to configure notification settings or to enable these. The closest thing is to be able to set up a custom notification subject/body, but this would not be useful if a notification cannot be configured.

Am I missing something in being confused why this capability is not there and is the only way truly to use the UI to configure notification destinations?

Has anyone identified a successful workaround to be able to fully set up an alert complete with notifications configured without any use of the UI?

Thanks in advance!

Re: Creating Databricks Alerts/Notifications without using the UI

Jerry01 — Tue, 08 Aug 2023 18:21:06 GMT

I am also running into a similar issue. Using databricks alert API, how to give refresh schedule, status, Email notifications, etc. I tried giving status as triggered but the alert is always getting created with unknown

Re: Creating Databricks Alerts/Notifications without using the UI

SaugatMukherjee — Tue, 12 Dec 2023 23:10:36 GMT

The way I hopefully figured this out (am going to try it out in a couple of days), was first doing it manually, getting an alert out to a teams channel (configured as a destination) , using the default template. When it actually wrote on teams, I got a link to a job (which apparently isn’t visible on the workflows page). Looking at the yaml of the job, it seems whenever a schedule is added to an alert, it creates a workflow with a “sql_task”.
Now, we have already been writing to teams channel, for failed jobs using email notifications and using the mail id of the channel (yes, there is an email link for a teams channel).

So, I would approach this as:

1. Create a saved query (through api, you can use terraform too).

2. Create an alert for the query. This only creates an alert with no schedule.

3. Create a workflow/job through api with a sql task pointing to an alertid and using email_notifications for the job (email notification can be used for both teams and to normal mailing groups).

I hope this should work, but I have to try it out and confirm in a few days.

This should however have to be simpler.

Re: Creating Databricks Alerts/Notifications without using the UI

SaugatMukherjee — Fri, 15 Dec 2023 15:45:47 GMT

I can now confirm that this works.

1. I created a saved query using databricks rest api.

2. Created an alert using databricks rest api. This does not include the schedule.

3. Created a job with a task of sql_task type with the alert id assigned to the alert created in step 2, again using the rest api.

While creating the job, you can also set email_notifications - that is also part of the payload if you look at the rest api definition for jobs.

Re: Creating Databricks Alerts/Notifications without using the UI

subbaram — Tue, 02 Jan 2024 07:24:12 GMT

Hi Saugat,

Is there a way that i can get the owner of a schema/table dynamically and send out a notification to him using Rest API.

Re: Creating Databricks Alerts/Notifications without using the UI

SaugatMukherjee — Wed, 03 Jan 2024 04:11:23 GMT

Hi Subbaram,

You should post a new question for this. I will then try to answer.

Re: Creating Databricks Alerts/Notifications without using the UI

meganst — Mon, 10 Jun 2024 13:46:18 GMT

Hi Saugat,

Just wondering how exactly you got the job url when you manually created the alert in the UI? I am calling to the jobs api and can't find any instances of the alert-scheduled sql query running.

Thanks!