cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Feature request: Run cluster in a different region

Erik
Valued Contributor II

We are one of those which unfortunately decided (earlier) to set up our infastructure in one of the sub-par Azure regions, and we happen to dont have any GPU clusters available.

We would like to test out some GPU enhanced ML in databricks, but at the moment our only solution is to create a completely new workspace in a new region and mount in our data there. This bypasses our CICD and our normal workflow, but works for testing.

I think it would be really usefull if we could create a cluster in another region from our normal workspace. I understand that it would cause extra network costs for both data transfer and managment, but it would allow us to run things in our normal workspace, and allow us to do the tradeoff between the extra cost of data-transfer vs the extra work of having multiple workspaces with multiple unity catalogs (since it can't span regions either).

It could very well be hidden behind a big red warning, maybe even a feature flag on the workspace level.

We need it for GPU clusters now, but honestly it would be quite usefull to also be able to use a serverless SQL warehouse seamlessly hosted in another region as well (with all the same caveats as above).

Alternatively, if unity catalog could span regions that would solve everything:-P

 

 

 

1 REPLY 1

Yeshwanth
Honored Contributor
Honored Contributor

@Erik 

I think it would be really useful if we could create a cluster in another region from our normal workspace

This does not seem like a possible thing for me as the Control plane resources and the Data plane resources are bound to that region. If it happens it would be great 🙂

Alternatively, if unity catalog could span regions that would solve everything:-P

Spot on!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!