Is it possible to get Azure Databricks cluster metrics using REST API thru pyspark code

[databricksdev]

Am trying to get azure  databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API by writing pyspark code. Its showing always cpu utilization & memory usage as N/A   where as data is showing in metrics data. 

[Ajay-Pandey · Accepted solution]

Hi @databricksdev  You can use System tables for Azure Databricks cluster metrics.
Please refer below blog for the same -
Compute system tables reference | Databricks on AWS