Brahmareddy
Esteemed Contributor II

Hi @DB1To3 , That makes sense. If Spark Connect is part of the next phase, starting with an all-purpose cluster sounds reasonable. In that case, I would focus on the Jobs API approach and map active runs back to the cluster ID. It would definitely be useful if Databricks provided a simpler cluster-level view of all active job runs directly.