I built Databricks Cost Observability — a full-stack observability dashboard
that runs natively as a Databricks App and reads directly from system tables.
No data leaves your workspace, no external BI tools needed.
What it covers across 11 tabs:
• Executive spend KPIs + AI_FORECAST() contract-year projection
• ML-powered anomaly detection (Isolation Forest) with cost impact
• Compute right-sizing — clusters, warehouses, DBR version sprawl
• Per-user query attribution with 30/60/90-day cost breakdown
• AI Gateway token usage by model (GPT-4, Claude, Mixtral, LLaMA, DBRX)
• Unity Catalog permission graph (interactive — users, groups, SPs, catalogs)
• Job SLA tracking — success rates, failure trends, DLT pipeline health
• Data lineage (bronze→silver→gold), MLflow, model registry
Works on free-tier workspaces via MOCK_MODE — no system table access needed.
Deploys in one git push via GitHub Actions + Databricks Asset Bundle.
GitHub: https://github.com/vijayakunuri1/databricks-cost-observability
Would love feedback — especially from AWS/GCP/Azure users. Happy to take feature
requests via GitHub Issues.
