Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a month ago
Hello Everyone,
We have a requirement to expose Databricks Unity Catalog Metric Views to custom applications running outside the Databricks environment.
The application team would like to consume governed KPIs and metrics through APIs such as REST or GraphQL, rather than accessing them directly through Databricks applications or SQL interfaces.
I’d like to understand the recommended Databricks architecture for this use case:
- Does Databricks provide an out-of-the-box REST or GraphQL API for querying Unity Catalog Metric Views from external applications?
- If not, is the recommended approach to build a custom API/service layer that queries the Metric Views through Databricks SQL?
- For applications requiring high concurrency and millisecond-level response times, would it make sense to materialize or synchronize the calculated metric data into Lakebase and expose it through an application/API layer?
- Would this introduce duplication between the governed Metric Views in Unity Catalog and the serving layer in Lakebase? If so, what is the recommended approach for keeping the metric definitions and serving data consistent?
- From an architecture perspective, which pattern would Databricks recommend:
Custom App → API → Databricks SQL → UC Metric Views
or
Custom App → API → Lakebase ← Metric Views / Databricks pipelines? - Are there other Databricks-native capabilities better suited for providing low-latency, highly concurrent access to governed metrics for external applications?
The main objective is to maintain Unity Catalog Metric Views as the governed semantic/metrics layer, while providing external applications with a scalable, low-latency API-based consumption model.
Regards,
Venkat
Venkat
Labels:
- Labels:
-
Partner