Is it possible to view Databricks cluster metrics using REST API

[Prashanthkumar]

I am looking for some help on getting databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API.

I am trying it in postman using databricks token and with my Service Principal bearer token with this link

https://databricksinstance.azuredatabricks.net/?o=workspaceid#setting/clusters/cluster_id/metrics

however when i send the request it shows below sign in, is there anything else i am missing or is it not possible to view using REST API

Here is the full response.

Any suggestions?

[Walter_C]

There is currently no option available to get this metrics available through API, but is coming soon.

[Prashanthkumar]

Thank you Walter will keep an eye for updates.

[ErikJ]

Is there a release calender where I can find a date for this?

[Nandhini_Kumar]

Is there any alternative way to get all performance metrics programmatically?

[ErikJ]

Through Databricks REST API you can get job info such as pipeline run time, task run times etc, but I don't think you can get CPU usage and such

[Prashanthkumar]

OK thank you, any plans to introduce a new feature in Databricks to capture CPU usage?

[javierbg]

At my company we are also interested in this feature, is there an ETA?

[moto-charles]

My company is also interested in this feature and interested in a roadmap or ETA.

[Walter_C]

There is no ETA yet available, as a temporary workaround you can get metrics details via the system tables through the node_timeline

https://docs.databricks.com/en/admin/system-tables/compute.html#node-timeline-table-schema 

[makulr]

Hard to believe this isnt a feature yet. I mean if I can go to the cluster, click metrics and see them, why can't I get them via API?? 

I guess I'll go with the workaround, but this is embarrassing honestly.

[Walter_C]

We understand and our product team is working on this, our direction has been to go through the system tables for this information but we are exploring in giving additional options