shan_chandra
Databricks Employee
Databricks Employee

@ivanychev  - Firstly, New workers are added and spark notice them hence, there is an init script logging in the event log stating the init script ran on the newly added workers.  For debugging, please check the Spark UI - executor tab. 

Secondly, For Spot Instance termination, This is mostly by the cloud provider and spot instance price fluctuation. you can ideally use hybrid clusters (with spot fall back on demand) flag set on the cluster configuration page. 

Reference: https://docs.databricks.com/en/compute/cluster-config-best-practices.html#on-demand-and-spot-instanc...

Thanks, Shan