cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

I have a job with strict performance requirements running in Azure Databricks. I had configured pools to reduce the startup time - but of late I'm seeing that the job cluster startup time is in the order of 5 to 8 minutes

sajith_appukutt
Honored Contributor II

Despite having it configured to run from pools, my Azure Databricks jobs cluster is taking 5 to 8 minutes to launch. Any pointers ?

1 ACCEPTED SOLUTION

Accepted Solutions

Ryan_Chynoweth
Honored Contributor III

If you are using pools then you should consider keeping a min idle count of machines greater than 2. This will allow you to have machines available and ready to use. If you have 0 machines on idle then the first job executed against the pool will have to wait for the machines to become available.

View solution in original post

1 REPLY 1

Ryan_Chynoweth
Honored Contributor III

If you are using pools then you should consider keeping a min idle count of machines greater than 2. This will allow you to have machines available and ready to use. If you have 0 machines on idle then the first job executed against the pool will have to wait for the machines to become available.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.