@Kirankumarbs 

If you jst want to track Job status Success/Failure, Run Duration, and Costs, use exporter that talks to the Databricks Workspace API or System Tables.

grafana/databricks-prometheus-exporter - This is an open-source exporter. It queries Databricks System Tables  to export metrics on billing, job runs, and SQL Warehouse performance. it needs a Service Principal with access to system.lakeflow