cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

AWS and Azure on the same Databricks workspace.

liamod
New Contributor II

Can I use compute instances from different providers i.e. AWS and Azure on the same Databricks workspace?

1 ACCEPTED SOLUTION

Accepted Solutions

Prabakar
Esteemed Contributor III

Hi @Liam ODonoghue​ there are a few methods using which you can connect AWS and Azure resources. In such cases, it involves only your accounts. But with Databricks, you need to handle two accounts for both cloud providers. Let's say if you create a workspace in AWS, you create the dataplane in your account and peer it with the control plane that is created in Databricks account. The same is applicable for Azure. If you want you can try VPN tunnel, ExpressRoute, site-to-site VPN to connect your cloud providers and share the resources. But when it comes to Databricks, making this setup is not possible as it has to go through a lot of security compliance and more research and testing to ensure there is no breach by any source. Might be the product and security team could consider this in the future as many enterprises go with hybrid cloud. You can raise a feature request via our ideas portal.

TLDR; This is not possible in Databricks for now.

View solution in original post

2 REPLIES 2

LearningDatabri
Contributor II

nice one. does databricks supports multi cloud ? this is not possible as the cloud vendors are different and both have different configuration.

Databricks folks @Jose Gonzalez​ @Prabakar Ammeappin​ @Abishek Subramanian​ @Kaniz Fatma​ can give you some input on this.

Prabakar
Esteemed Contributor III

Hi @Liam ODonoghue​ there are a few methods using which you can connect AWS and Azure resources. In such cases, it involves only your accounts. But with Databricks, you need to handle two accounts for both cloud providers. Let's say if you create a workspace in AWS, you create the dataplane in your account and peer it with the control plane that is created in Databricks account. The same is applicable for Azure. If you want you can try VPN tunnel, ExpressRoute, site-to-site VPN to connect your cloud providers and share the resources. But when it comes to Databricks, making this setup is not possible as it has to go through a lot of security compliance and more research and testing to ensure there is no breach by any source. Might be the product and security team could consider this in the future as many enterprises go with hybrid cloud. You can raise a feature request via our ideas portal.

TLDR; This is not possible in Databricks for now.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group