cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

I built a free 11-tab Cost Observability Dashboard as a Databricks App โ€” open source

vijayakunuri1
New Contributor III

I built Databricks Cost Observability โ€” a full-stack observability dashboard
that runs natively as a Databricks App and reads directly from system tables.
No data leaves your workspace, no external BI tools needed.

What it covers across 11 tabs:
โ€ข Executive spend KPIs + AI_FORECAST() contract-year projection
โ€ข ML-powered anomaly detection (Isolation Forest) with cost impact
โ€ข Compute right-sizing โ€” clusters, warehouses, DBR version sprawl
โ€ข Per-user query attribution with 30/60/90-day cost breakdown
โ€ข AI Gateway token usage by model (GPT-4, Claude, Mixtral, LLaMA, DBRX)
โ€ข Unity Catalog permission graph (interactive โ€” users, groups, SPs, catalogs)
โ€ข Job SLA tracking โ€” success rates, failure trends, DLT pipeline health
โ€ข Data lineage (bronzeโ†’silverโ†’gold), MLflow, model registry

Works on free-tier workspaces via MOCK_MODE โ€” no system table access needed.
Deploys in one git push via GitHub Actions + Databricks Asset Bundle.

GitHub: https://github.com/vijayakunuri1/databricks-cost-observability

Would love feedback โ€” especially from AWS/GCP/Azure users. Happy to take feature
requests via GitHub Issues.

1 ACCEPTED SOLUTION

Accepted Solutions

Got a new feature added to include cloud cost (multi-cloud)

vijayakunuri1_1-1777522882386.png

 

 

View solution in original post

3 REPLIES 3

cyborg-de
New Contributor II

That is really good, would be better to get platform cost in it as well, like Azure costing for subscription triggered using cost api and saved to databricks tables. 

Got a new feature added to include cloud cost (multi-cloud)

vijayakunuri1_1-1777522882386.png

 

 

vijayakunuri1
New Contributor III

The main tradeoff: Azure Cost Management API has a 1-2 day lag, so the data is never truly real-time, but that's fine for a cost observability use case. We can have a ingestion notebook would run daily to get those cost ingested. 
Azure Cost Management โ†’ Databricks tables โ†’ Dashboard tab