Serverless Compute - Python - Custom Emails via SMTP (smtplib.SMTP(host_name)) - Any alternative?

[Ramana]

Hello Community,

We have been trying to migrate our jobs from Classic Compute to Serverless Compute. As part of this process, we face several challenges, and this is one of them.

We have several scenarios where we need to send an inline email via Python Code and then process to execute the rest of the code or make the job fail. We know that Databricks workflows have failure/success/threshold email functionality, but that is not helpful for my scenarios.

In Classic Compute, I can send the emails by using smtplib.SMTP(host_name)/

sendmail(), but when it comes to the Serverless, this is not working.

Is there an alternative to sending code in-line emails when we execute any job with Serverless Compute?

#SMTP

#Custom-Email-Functionality

#Python

#Serverless-Compute

#Workflows

[szymon_dybczak]

Hi @Ramana ,

What error do you get in serverless? Could you provide error message?

[Ramana]

@szymon_dybczak it is neither a code issue nor an SMTP host issue, but it is purely related to Serverless Compute.

The same code and hostname work perfectly fine on Classic Compute but not on Serverless.

I don't think Databricks Serverless can recognize the third-party/client hostnames. If not, at least Databricks should have an alternative hostname to send custom emails.

Here is my code snippet and error details.

import email
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import traceback
import logging

# Intentionally erased all parameters before posting the code here
to_email = ""
from_email = ""
subject = "Serverless Email!"
message = "Serverless Email!"
cc_email = ""
bcc_email = ""
host_name = ""

try:
    print("Sending email - TO: " + to_email)
    msg = MIMEText(message)
    msg["Subject"] = subject
    msg["Message-id"] = email.utils.make_msgid()
    msg["From"] = from_email
    msg["To"] = to_email
    to_email = to_email.split(",")
    if cc_email:
        print("Sending email - CC: " + cc_email)
        msg["Cc"] = cc_email
        cc_email = cc_email.split(",")
        to_email = to_email + cc_email
    if bcc_email:
        # msg["Bcc"] = bcc_email
        bcc_email = bcc_email.split(",")
        to_email = to_email + bcc_email
    server = smtplib.SMTP(host_name)
    server.sendmail(from_email, to_email, msg.as_string())
    server.quit()
    print("Email Sent Successfully!")
except smtplib.SMTPException as e:
    print("Email Sending Failed.\nSMTP Exception: <br>" + str(e))
    # logging.error(e)
    traceback.print_exc()
    raise Exception(f"Exception occurred: {e}")
except Exception as e:
    print("Email Sending Failed. Exception: <br>" + str(e))
    # logging.error(e)
    traceback.print_exc()
    raise Exception(f"Exception occurred: {e}")

Sending email - TO: 
Email Sending Failed. Exception: <br>[Errno -2] Name or service not known
Traceback (most recent call last):
  File "/home/spark-1cd5f4c8-1a9c-413f-963c-1a/.ipykernel/2497/command-8820583165797855-46947889", line 34, in <module>
    server = smtplib.SMTP(host_name)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/socket.py", line 828, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/socket.py", line 963, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known