is Spark UI available on the Databricks Free Edition?

[Nietzsche]

Hi all

I have a noob question, I am currently using the Databricks free edition, which runs on serverless compute.

To access the Spark UI normally one would click on the attached compute, however, with serverless, I can not find the menu to access Spark UI.

I want to see the DAG visualization, is this possible in the Databricks Free Edition?

[sridharplv · Accepted solution]

Databricks Free Edition runs on serverless compute due to which we do not have access to the Spark UI, including the DAG visualization.

Spark UI (with DAG, stages, tasks, jobs, and executor tabs) is traditionally accessed via attached interactive clusters. Serverless compute, as used in the Free Edition, abstracts the underlying infrastructure. You don’t get visibility into low-level cluster details like executor memory, task distribution, or Spark DAGs. The "Compute" tab, where you would normally click into a cluster to open the Spark UI, is not available or limited in the Free Edition.

[Nietzsche]

@sridharplv 

Thank you for the clarification, I expected this might be the case.

Ben.

[dyusuf]

So, there is no way we can run spark in free edition as we need general purpose clusters?