Hi, 

I have solved the problem with the same workers and driver.

In my case data skewness was the problem.

Adding repartition to the dataframe just before writing, evenly distributed the data across the nodes and this stage failure resolved.

Thanks @Retired_mod for your insoghts.