Azure Databricks Serverless private connection to S3 bucket

mgcasas-aws
New Contributor

I'm looking for technical references to connect an Azure Databricks serverless workspace to an S3 bucket over a private site-to-site VPN connection.

 

Found the following to connect AWS (consumer) to Azure (provider), but I'm looking for the other way.

https://community.databricks.com/t5/technical-blog/private-cross-cloud-delta-sharing-with-serverless...

Sai_Ponugoti
Databricks Employee
Databricks Employee

Hello @mgcasas-aws 
Thank you for your question!

We’re currently working on a solution for private cross-cloud Delta Sharing (Azure → AWS).

In the meantime, here’s a possible approach:

  • Update your Azure Storage Account network settings from private endpoints to “allow access from selected networks.”

  • From AWS Serverless, you can then obtain a stable NAT IP (PuPr) and whitelist those IPs on your Azure Storage Account.

This way, you can continue to leverage AWS Serverless compute while keeping your data in Azure private.
Please note: you’ll need to be enrolled in the preview to access NAT IPs for AWS Serverless.

Sai Ponugoti

View solution in original post

Venkatauppuluri
New Contributor II

Hello @Sai_Ponugoti any progress on the solution?