nengen
Databricks Partner

I have a pretty complex and large SQL query which does a lot of joins on CTEs. Due to the nature of the data this has to be done using cross joins so I suspect that this might be the reason it is slow. I was hoping to be able to pinpoint where the tasks are waiting for available nodes or where the query is taking so much time (wall clock duration). I tried using the query profiler but this seems to show the execution time of the tasks and not the whole process.