cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to get Azure Databricks cluster metrics using REST API thru pyspark code

databricksdev
New Contributor II

Am trying to get azure  databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API by writing pyspark code. Its showing always cpu utilization & memory usage as N/A   where as data is showing in metrics data. 

1 ACCEPTED SOLUTION

Accepted Solutions

Ajay-Pandey
Esteemed Contributor III

Hi @databricksdev  You can use System tables for Azure Databricks cluster metrics.
Please refer below blog for the same -
Compute system tables reference | Databricks on AWS

View solution in original post

1 REPLY 1

Ajay-Pandey
Esteemed Contributor III

Hi @databricksdev  You can use System tables for Azure Databricks cluster metrics.
Please refer below blog for the same -
Compute system tables reference | Databricks on AWS