Hi @erwingm10 ,

Unfortunately, currently that there is no direct endpoint in REST API to get cluster metrics. You can extract some ganglia metrics through custom scripting, but they're not so detailed like the one you looking for.

Look at below links how to implement one:

https://medium.com/@alexandr.volok/databricks-observability-collecting-ganglia-metrics-d2848d5944ee

https://alexvolok.com/2023/collecting-databricks-clusters-metrics/

But support for Cluster Level Metrics API is planned feature according to below thread, so in near future we can expect such an option:

https://community.databricks.com/t5/get-started-discussions/is-it-possible-to-view-databricks-cluste...