Hello Community,

I’m exploring ways to export the default metrics from Databricks Jobs/Workflows to Prometheus and visualize them in Grafana.

There’s a nice option to use Databricks as a data source in Grafana, but that’s available only in the Enterprise edition. Before building a custom script or embedding logic directly into my Spark jobs, I’d like to check if there’s an open-source alternative.

Are there any recommended open-source tools, libraries, or init scripts that can help with this?