- 2684 Views
- 3 replies
- 0 kudos
Resolved! Frequent spot loss of driver nodes resulting in failed jobs when using spot fleet pools
When using spot fleet pools to schedule jobs, driver and worker nodes are provisioned from the spot pools and we are noticing jobs failing with the below exception when there is a driver spot loss. Share best practices around using fleet pools with 1...
- 2684 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
In this scenario, the driver node is reclaimed by AWS. Databricks started preview of hybrid pools feature which would allow you to provision driver node from a different pool. We recommend using on-demand pool for driver node to improve reliability i...
- 0 kudos