cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to get Azure Databricks cluster metrics using REST API thru pyspark code

databricksdev
New Contributor II

Am trying to get azure  databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API by writing pyspark code. Its showing always cpu utilization & memory usage as N/A   where as data is showing in metrics data. 

1 ACCEPTED SOLUTION

Accepted Solutions

Ajay-Pandey
Esteemed Contributor III

Hi @databricksdev  You can use System tables for Azure Databricks cluster metrics.
Please refer below blog for the same -
Compute system tables reference | Databricks on AWS

Ajay Kumar Pandey

View solution in original post

1 REPLY 1

Ajay-Pandey
Esteemed Contributor III

Hi @databricksdev  You can use System tables for Azure Databricks cluster metrics.
Please refer below blog for the same -
Compute system tables reference | Databricks on AWS

Ajay Kumar Pandey

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now