What is overwatch ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2021 03:57 AM
I heard Databricks recommend overwatch for monitoring Clusters, Can anybody help like what all metrics it will provide that , how it is helpful in monitoring or better than ganglia ?
- Labels:
-
Data Ingestion & connectivity
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2021 04:33 AM
Overwatch is a different kind of tool - right now it couldn't be used for real-time monitoring, like, Ganglia. Overwatch collects data from the multiple data sources (audit logs, APIs, cluster logs, etc.), process, enrich and aggregate them following traditional Bronze/Silver/Gold approach. The data that is provided by Overwatch could be used for different purposes:
- Cost estimation - it may provide more granular analysis, like, attributing costs to specific notebooks and users, and also overcome the limits for clusters acquired from the instance pools
- Governance and monitoring with much longer periods of time and much cheaper compared to Azure Log Analytics or other solutions
- Identifying the problematic workloads & optimizing them
- ...
Overwatch is available to Databricks customers as part of the Databricks Labs. Follow the documentation for installation instructions.