topic Re: AWS Email sending challenge from Databricks with UNITY CATALOG and Multinode cluster in Data Engineering

AWS Email sending challenge from Databricks with UNITY CATALOG and Multinode cluster

DataEngineer — Fri, 17 May 2024 19:05:11 GMT

Hi,

I have implemented the UNITY CATALOG with multinode cluster in databricks. The workspace instance profile with EC2 access is also created in IAM. but still having a challenge in sending emails from databricks using SES service.

The same is working fine with single node cluster. Any inputs would be much appreciated.

Instance profile creation : Manage instance profiles in Databricks | Databricks on AWS

Re: AWS Email sending challenge from Databricks with UNITY CATALOG and Multinode cluster

Ayushi_Suthar — Mon, 20 May 2024 06:03:11 GMT

Hi @DataEngineer , Good Day!

Can you please confirm what challenges you are facing while sending emails so that we can help you accordingly?

Also, can you please check the notebook below, which demonstrates how to send an email message using Amazon SES?

Notebook: https://docs.databricks.com/en/_extras/notebooks/source/kb/notebooks/send-email-aws.html

You can refer to this KB article as well: https://kb.databricks.com/en_US/notebooks/send-email

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi

Re: AWS Email sending challenge from Databricks with UNITY CATALOG and Multinode cluster

Babu_Krishnan — Thu, 23 May 2024 13:34:19 GMT

Hi @DataEngineer ,

Are you able to resolve the issue. We are having the same issue when we try to use MultiNode cluster for UnityCatalog. Email functionality was working fine with Single node cluster.

We are getting "ConnectionRefusedError: [Errno 111] Connection refused" error.