topic any open-source library or init script for exporting Databricks job metrics to Prometheus in Data Engineering

any open-source library or init script for exporting Databricks job metrics to Prometheus

Kirankumarbs — Wed, 28 Jan 2026 08:55:01 GMT

Hello Community,

I’m exploring ways to export the default metrics from Databricks Jobs/Workflows to Prometheus and visualize them in Grafana.

There’s a nice option to use Databricks as a data source in Grafana, but that’s available only in the Enterprise edition. Before building a custom script or embedding logic directly into my Spark jobs, I’d like to check if there’s an open-source alternative.

Are there any recommended open-source tools, libraries, or init scripts that can help with this?

Re: any open-source library or init script for exporting Databricks job metrics to Prometheus

Raman_Unifeye — Wed, 28 Jan 2026 09:24:27 GMT

@Kirankumarbs

If you jst want to track Job status Success/Failure, Run Duration, and Costs, use exporter that talks to the Databricks Workspace API or System Tables.

grafana/databricks-prometheus-exporter - This is an open-source exporter. It queries Databricks System Tables to export metrics on billing, job runs, and SQL Warehouse performance. it needs a Service Principal with access to system.lakeflow

Re: any open-source library or init script for exporting Databricks job metrics to Prometheus

Kirankumarbs — Wed, 28 Jan 2026 10:02:13 GMT

I checked library documentation! It seems good start for me to visualize System tables at Grafana!