topic Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to utilize databricks existing smtp or databricks api. in Data Engineering

Send custom emails from databricks notebook without using third party SMTP server. Would like to utilize databricks existing smtp or databricks api.

969091 — Wed, 29 Mar 2023 09:53:41 GMT

We want to use existing databricks smtp server or if databricks api can used to send custom emails. Databricks Workflows sends email notifications on success, failure, etc. of jobs but cannot send custom emails. So we want to send custom emails to differentiate databricks environment whether email notifications are received from Dev, QA, Stage or Prod environment.

What are other possible ways to send emails? Please advise.

I have used python smtplib package to send message but got error ConnectionRefusedError: [Errno 111] Connection refused at smtpObj = smtplib.SMTP('localhost')

import smtplib

sender = 'abc@gcd.com'

receivers = ['myname@company.com']

message = """From: From Person <abc@gcd.com>

To: To Person <myname@company.com>

Subject: SMTP e-mail test

This is a test e-mail message.

"""

try:

smtpObj = smtplib.SMTP('localhost')

smtpObj.sendmail(sender, receivers, message)

print ("Successfully sent email")

except SMTPException:

print ("Error: unable to send email")

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to utilize databricks existing smtp or databricks api.

LandanG — Wed, 29 Mar 2023 15:11:18 GMT

@Krishna Prasad Depending on the emails and how you want them to be sent, you could check out Databricks Alerts where you can send alerts depending on custom logic with a Delta table. You could use Jobs to combine the notebooks and SQL alerts to raise alerts if a notebook failed or something.

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to utilize databricks existing smtp or databricks api.

Anonymous — Thu, 30 Mar 2023 03:51:21 GMT

Hi @Krishna Prasad

Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.

Please help us select the best solution by clicking on "Select As Best" if it does.

Your feedback will help us ensure that we are providing the best possible service to you.

Thank you!

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

htv — Wed, 23 Oct 2024 06:14:46 GMT

Currently, the Alert service only allow to send a fix email list. can we send dynamic based on the a column in the dataset ? e.g. i have a dataset which includes a email column, can i send email to each row based on columns ?

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

pavlos_skev — Mon, 24 Feb 2025 10:50:15 GMT

Hello. Did you figure this out? We have a similar problem and this approach could potentially be a nice solution.

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

mido1978 — Thu, 27 Feb 2025 17:06:53 GMT

So you could do this via the api and combine with alerts. The challenge is obtaining the query_ids needed to create the notification and the notification destinations themselves - it would go something like this -

- Save your queries as actual "queries" in Databricks
- Create a table storing your metadata, so the query display name, email recipients, a column for the trigger column and a column for the query_id

- Use the notification_destination api, iterate through your table and create a destination per email in your table
- Use the queries api, iterate through the queries you own using

queries.list() - update your metadata table query_id by joining on display_name and updating the query_id. Note, you can combine this step with the next by doing the queries_create() api at the same time if your queries themselves are stored in a table. This is quite intensive so it might be better to create the queries via the api and retrieve the query_id at the same time
- Now you've got your query_ids and notification defined, you can then iterate through your table to create your actual alerts, passing through your destinations and queries that you've now defined, there's an example here

Might sound a bit clunky - if you're happy to wait i'm building a POC for a customer as we speak doing this very thing, if you're happy to wait a few days i'll share the final code here 🙂

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

mido1978 — Wed, 19 Mar 2025 09:15:10 GMT

Hey - i ended up writing a blog on this, hopefully a simpler way of doing things than I thought 🙂

https://sqlofthenorth.blog/2025/03/04/data-driven-notifications-with-databricks/

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

pk13 — Wed, 19 Mar 2025 09:17:23 GMT

Hello @mido1978

I am also in need of something similar.

I have two tables one has the details of the recipient and another has some log data which are identified by a common key. Emails need be sent with the mentioned log to the recipient for whom it's applicable.

Please share the POC/code when it's ready🙂.

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

mido1978 — Wed, 19 Mar 2025 10:07:11 GMT

hey @pk13 see my reply this morning, hope this is of use for you?

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

JohnnyWong120 — Tue, 24 Jun 2025 15:45:02 GMT

I saw your code from the link you provided. But it is using the third party smtp server. Is there any function that directly from the Databrick it was hosted? I can use the notification of job to send standard success fail email. How that function to send the email? Thank you so much!

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

JR-20773 — Fri, 08 Aug 2025 14:59:13 GMT

Any update on the original request of not using 3rd party smtp server for custom emails? Can that be done outside of Alerts or the default Job Notifications?

Re: Send custom emails from databricks notebook without using third party SMTP server. Would like to

Shivaprasad — Thu, 18 Dec 2025 17:14:05 GMT

Did you able to get the custom email working from databricks notebook. I was trying but was not successful. let me know