cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Can I schedule Databricks pools to have different minimum idle instance counts at different times of the day

sajith_appukutt
Honored Contributor II

I have few jobs configured to run against a pool at 10 PM every night. After running some tests, I found that increasing minimum idle instance counts improves the job latencies. However, It wouldn't be needed to have so many VMs idle at other times. Is it possible to schedule the pool to have different minimum idle instance counts at different times of the day

1 ACCEPTED SOLUTION

Accepted Solutions

Ryan_Chynoweth
Honored Contributor III

Yes you can do so programmatically using the REST APIs. You can edit the settings of a Databricks Pool by using the Instance Pool Edit endpoint and provide the min idle that you desire. This cannot be done via the web UI.

View solution in original post

1 REPLY 1

Ryan_Chynoweth
Honored Contributor III

Yes you can do so programmatically using the REST APIs. You can edit the settings of a Databricks Pool by using the Instance Pool Edit endpoint and provide the min idle that you desire. This cannot be done via the web UI.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.