Streamlining Custom Job Notifications with a Centralized Email List

[Raj_DB]

Hi Everyone,

I am working on setting up success/failure notifications for a large number of jobs in our Databricks environment. The manual process of configuring email notification using UI for each job individually is not scalable and is becoming very time-consuming.

To solve this, I am exploring a centralized approach. The goal is to define a list of email recipients in a single location (like a notebook or a table) and have all jobs reference this central list for their notifications.

Could you please suggest the best practices or available methods for achieving this? 

Thank you for your assistance.

[mmayorga · Accepted solution]

Hi @Raj_DB ,

Thank you for reaching out!

You can easily achieve this by leveraging the Python SDK that is already installed within the Databricks clusters or by using the Jobs API.

With the SDK, you can update each job and its corresponding “email_notifications”, using the code below:

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.jobs import JobSettings, JobEmailNotifications

w = WorkspaceClient()
w.jobs.update(
    job_id=<JOB_ID>,
    new_settings=JobSettings(
        email_notifications=JobEmailNotifications(
            on_start=["<EMAIL_HERE>"],
            on_success=["<EMAIL_HERE>"],
            on_failure=["<EMAIL_HERE>"]
        )
    )
)

You can extend this code by previously querying the list() function or providing a predefined list of Jobs IDs and navigating through that list and update their notifications. You can also create a json with mapping of your jobs and their corresponding emails while using this code snippet.

Remember that you can also set Notifications at the Task level.

I hope this helps!, If this solution works for you, please click the "Accept as Solution" button.

[Raj_DB]

Thank you @mmayorga, The solution you provided worked perfectly.

However, we did hit one small snag: the notification works for individual email addresses, but it doesn't seem to be sending to our team's group email (e.g., data-team@company.com). Is this a known limitation in Databricks, or is there a workaround we might have missed?

[mmayorga]

Hi @Raj_DB 

As far as I understand, this should work with individual or email groups. In the documentation for the JobEmailNotifications, I don't see anything that could disrupt its behavior . Here are the specific considerations about the behavior of the notifications but none related to email groups.

I'm curious: if you set the job notification manually via the UI using the email group, are you able to receive the email? If you do then, I may suggest upgrading the SDK.

%pip install --upgrade databricks-sdk
dbutils.library.restartPython()

 I hope this helps!

Thank you Raj!

[Raj_DB]

Hi @mmayorga, Just to clarify, I have also tried adding the group email directly through the Databricks Jobs UI, and I'm still not receiving notifications there either.

[mmayorga]

hey @Raj_DB , thanks for the quick response. On my end, I've tested and confirmed that it works with email groups.

May I suggest in checking if any filter rule is applied to that email group?

Thank you!

[Raj_DB]

Hi @mmayorga, I will check and thank you so much for your guidance, I really appreciate your help.

[nayan_wylde]

@Raj_DB Databricks sends notifications via its internal email service, which often requires the address to be a valid individual mailbox or a distribution list that accepts external mail. If your group email is a Microsoft 365, Please check if 
“Allow external senders” is enabled. No moderation or approval is blocking the email.

Try adding your personal email alongside the group email in the same job:

"email_notifications": {
  "on_failure": ["your.email@company.com", "group@company.com"]
}

[Raj_DB]

Thank you @nayan_wylde , I will check.