shyam_9
Databricks Employee
Databricks Employee

Hi @naveen reddy

If you have 3 nodes with 32 GB memory specified each you have just 30 GB for everything else, the different overheads add up quick and it's entirely possible that this is too little and the executors get killed for hogging the memory.

Try using something like 24 GB per node or just play around with the values.