Databricks Spot Instance: Completion Guarantee

[a_user12]

Databricks allows to use spot instances for worker nodes.

I consider to use them for interactive clusters. Do I have a gurantee that code will be completed without any errors even if spot instances are evicted? I would accept execution delays but no errors.

Here, it is just stated:

Spot instances are a good choice for workloads where it is acceptable to take longer because one or more spot instances have been evicted by the cloud provider.

[Giri-Patcham · Accepted solution]

Hi,

A job might fail when spot instances are evicted immediately or more frequently during shuffle operations.

Thanks