cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Azure Databricks Serverless private connection to S3 bucket

mgcasas-aws
New Contributor

I'm looking for technical references to connect an Azure Databricks serverless workspace to an S3 bucket over a private site-to-site VPN connection.

 

Found the following to connect AWS (consumer) to Azure (provider), but I'm looking for the other way.

https://community.databricks.com/t5/technical-blog/private-cross-cloud-delta-sharing-with-serverless...

1 REPLY 1

Sai_Ponugoti
Databricks Employee
Databricks Employee

Hello @mgcasas-aws 
Thank you for your question!

Weโ€™re currently working on a solution for private cross-cloud Delta Sharing (Azure โ†’ AWS).

In the meantime, hereโ€™s a possible approach:

  • Update your Azure Storage Account network settings from private endpoints to โ€œallow access from selected networks.โ€

  • From AWS Serverless, you can then obtain a stable NAT IP (PuPr) and whitelist those IPs on your Azure Storage Account.

This way, you can continue to leverage AWS Serverless compute while keeping your data in Azure private.
Please note: youโ€™ll need to be enrolled in the preview to access NAT IPs for AWS Serverless.

Sai Ponugoti