User16783853906
Databricks Employee
Databricks Employee

We started the Private Preview of AWS EC2 Fleet from early 2021. AWS EC2 Fleet feature allows customers to create Databricks pools and get VMs of multiple AZs and multiple instance families & sizes. When clusters are created from the Databricks EC2 fleet pools, they gain the benefit of even lower EC2 spot price and more instance availability. Specifically:

  • Clusters created from EC2 Fleet pool will get VMs from the cheapest AZ within your pool definition.
  • They will also get VMs of the cheapest type within your pool definition.
  • When clusters experience a spot kill, instead of falling back to on-demand VMs, the pool will look for another spot VM from either a different AZ or another instance type.

View solution in original post