Spark UI SQL/Dataframe tab missing queries

kori73
New Contributor

Hi,

 

Recently, I am having some problems viewing the query plans in the Spark UI SQL/Dataframe tab.

 

I would expect to see large query plans in the SQL tab where we can observe the details of the query such as the rows read/written/shuffled. However, I see just a single query with no details. The following screenshots might be helpful to understand the issue.

 

image (10)Here, I was expecting to see a large query plan (I can confirm this by looking at the Jobs tab).

image (7) (1) 

I am having this issue in multiple Databricks Runtimes such as DBR 9.1 LTS, DBR 10.4 LTS , and DBR 12.2.

 

Any help is much appreciated!