cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Feature request: Run cluster in a different region

Erik
Valued Contributor III

We are one of those which unfortunately decided (earlier) to set up our infastructure in one of the sub-par Azure regions, and we happen to dont have any GPU clusters available.

We would like to test out some GPU enhanced ML in databricks, but at the moment our only solution is to create a completely new workspace in a new region and mount in our data there. This bypasses our CICD and our normal workflow, but works for testing.

I think it would be really usefull if we could create a cluster in another region from our normal workspace. I understand that it would cause extra network costs for both data transfer and managment, but it would allow us to run things in our normal workspace, and allow us to do the tradeoff between the extra cost of data-transfer vs the extra work of having multiple workspaces with multiple unity catalogs (since it can't span regions either).

It could very well be hidden behind a big red warning, maybe even a feature flag on the workspace level.

We need it for GPU clusters now, but honestly it would be quite usefull to also be able to use a serverless SQL warehouse seamlessly hosted in another region as well (with all the same caveats as above).

Alternatively, if unity catalog could span regions that would solve everything:-P

 

 

 

1 REPLY 1

Yeshwanth
Databricks Employee
Databricks Employee

@Erik 

I think it would be really useful if we could create a cluster in another region from our normal workspace

This does not seem like a possible thing for me as the Control plane resources and the Data plane resources are bound to that region. If it happens it would be great 🙂

Alternatively, if unity catalog could span regions that would solve everything:-P

Spot on!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group