cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

How do Databricks handle the utilization of Azure Reserved Instances?

NathanL
New Contributor II

We are trying to use Azure Reserved Instances for Databricks clusters as we have some consistent workload.

  1. Does Databricks automatically select the reserved instances when they are available (assuming we keep the on-demand instance type in the cluster config)? Or is there explicit configuration we need to set to have Databricks use those reserved instances?
  2. Does SQL Warehouse support using Reserved Instances? I have seen 2 other posts around using Reserved Instances for general compute cluster or in instance pool (Optimize costs for your Data and AI workloads with... - Databricks Community - 66241Does Databricks support instance pools of Azure re... - Databricks Community - 18090) but can't really find any documentation around reserved instances or SQL warehouse support.

 

1 REPLY 1

NathanL
New Contributor II

Thanks for your response @Retired_mod. I did find the DBU prepurchase plan but that seems to be separated from the "Azure Reserved Instances" concept. My understanding is that there are 2 bills when using Azure Databricks, one for Databricks itself which accounts for the DBU costs, one for the other infrastructure (i.e. VMs) on Azure. Prepurchase can lowers the DBU costs, but Reserved Instances intends to lower the VM costs.

I did see the post mentioned "Databricks supports instance pools that can come from your reserved instances" but want to see if there's additional setup needed to enable that, and if spinning up a cluster can automatically pick up the Reserved Instances without creating a pool.

Thanks!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group