Dooley
Databricks Employee
Databricks Employee

Thanks for the files.

(1) So in this Databricks Shell - Details for Query file, in the graph you need to expand the blocks and see if you see any data spill in there.

(2) Can you please submit the query that you were doing that ran into this problem? How much data are you running this on? Is that data partitioned by you in a custom way?

Side note: I did see in one of the stages you are doing a count distinct and I wanted to draw your attention to a Approx Count Distinct SQL function.