- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-28-2026 12:55 AM
Hello Community,
I’m exploring ways to export the default metrics from Databricks Jobs/Workflows to Prometheus and visualize them in Grafana.
There’s a nice option to use Databricks as a data source in Grafana, but that’s available only in the Enterprise edition. Before building a custom script or embedding logic directly into my Spark jobs, I’d like to check if there’s an open-source alternative.
Are there any recommended open-source tools, libraries, or init scripts that can help with this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-28-2026 01:24 AM
If you jst want to track Job status Success/Failure, Run Duration, and Costs, use exporter that talks to the Databricks Workspace API or System Tables.
grafana/databricks-prometheus-exporter - This is an open-source exporter. It queries Databricks System Tables to export metrics on billing, job runs, and SQL Warehouse performance. it needs a Service Principal with access to system.lakeflow
RG #Driving Business Outcomes with Data Intelligence
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-28-2026 02:02 AM
I checked library documentation! It seems good start for me to visualize System tables at Grafana!
