Cluster compute metrics

[User1234]

I want to fetch compute metrics(hardware, gpu and spark) and use them in certain dashboard on Databricks, however i'm not able to fetch them. i have tried GET API request and system tables. The system tables only have CPU utilization and memory utilization per node but i want that on the granular level as shown in the graphs in compute metrics screen. How can i fetch these details and download them in json or csv format?

[Kaniz_Fatma]

Hi @User1234, To fetch compute metrics in Databricks, you can use the native compute metrics tool available in the Databricks UI.

Here’s how you can access and utilize these metrics:

1. Access Compute Metrics UI:

   • Click on Compute in the sidebar.
   • Choose the specific compute resource (cluster) for which you want to view metrics.
   • Click the Metrics tab.
   • By default, hardware metrics are displayed. To view Spark metrics, click the drop-down menu labeled Hardware and select Spark. If the instance is GPU-enabled, you can also select GPU¹.

2. Granularity and Time Period:

   • Metrics are available in almost real-time with a normal delay of less than one minute.
   • You can view historical metrics by selecting a time range using the date picker filter. Metrics are collected every minute, so you can filter by any range of day, hour, or minute from the last 30 days.
   • Click the calendar icon to select from predefined data ranges or define custom values¹.

3. Node-Level Metrics:

   • You can view metrics for individual nodes by clicking the Compute drop-down menu and selecting the specific node you’re interested in.
   • Note that GPU metrics are only available at the individual-node level, while Spark metrics are not a...¹.

4. Available Hardware Metrics:

   • The following hardware metric charts are available in the compute metrics UI:
     • Server Load Distribution: Shows CPU utilization over the past minute for each node.
     • CPU Utilization: Displays the percentage of time the CPU spent in different modes (e.g., guest, iowait, idle, irq, nice, softirq, steal).
     • Memory Utilization: Provides insights into memory usage.
     • Filesystem Metrics: Includes disk space usage.
     • Network Metrics: Covers network activity.
   • These metrics can help you monitor resource usage and identify performance bottlenecks¹.

5. Exporting Metrics:

   • Unfortunately, the native compute metrics UI does not directly support exporting metrics to JSON or CSV format.
   • However, you can manually record the relevant metrics or use third-party tools to automate data extraction from the UI.

Remember that these metrics provide valuable insights into your cluster’s resource usage, and they can help you optimize performance and troubleshoot errors. If you need more granular data, consider using additional monitoring tools or integrating with exter...

[User1234]

Hey thanks for the response. I'm aware that these metrics are available in compute metrics UI but I wanted the data which I can export and use for analysis. is there any way to get the data which is being used to create these graphs, manually recording is difficult because data keeps getting updated every hour. Could you please provide any insights on external tools that can be used to automate data extraction from UI?

[DPG1112]

I am also curious to see if there is any alternative to programatically (and not just from the UI) fetch this data. This would be highly valuable. Thanks!

[BlakeWood]

Replying for the updates.

[mousie_groove]

Can you tell what 3rd party tools you are referring to?