Prabakar
Databricks Employee
Databricks Employee

hi @Boumaza nadia​ Please check the Ganglia metrics for the cluster. This could be a scalability issue where cluster is overloading. This can happen due to a large partition not fitting into the given executor's memory. To fix this we recommend bumping up the worker node type. Switch to a bigger worker node instance to mitigate the issue.

View solution in original post