mark_ott
Databricks Employee
Databricks Employee

I'm guessing you are running one of more Wide transformations in your query and that is causing Skewed Shuffle Partitions.  Go back to Stages tab and check out 'Shuffle Write Size/Records' row.

m2.png