- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-22-2022 02:46 AM
Does anyone have experience with the mspnp/spark-monitoring library ?
Is this best practice, or are there better ways to monitor a Databricks Cluster?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-22-2022 03:06 AM
@Michael Galli , Yes, exactly mspnp/spark-monitoring library for Azure Monitor for that https://docs.microsoft.com/en-us/azure/architecture/databricks-monitoring/application-logs
I definitely prefer datadog https://www.datadoghq.com/blog/databricks-monitoring-datadog/. It is simple to integrate and more straightforward to visualize.
My blog: https://databrickster.medium.com/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-22-2022 03:55 AM
@Hubert Dudek Nice... Is it also possible to monitor the metrics collected with mspnp/spark-monitoring in datadog?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-25-2022 08:38 AM
@Michael Galli I don't think you can monitor metrics captured by mspnp/spark-monitoring in datadog, there is a service called Azure Log Analytics workspace where these logs are available for querying.
You can also check out below if you are interested.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2022 04:25 AM
It´s resolved for me, thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2025 02:50 PM
Interesting that Microsoft deleted this project. Was there any announcement as to when, why, and what to do now?
