topic Re: cluster metrics databricks runtime 13.1 in Data Engineering

cluster metrics databricks runtime 13.1

seefoods — Wed, 10 Jan 2024 08:20:14 GMT

hello everyone,
how to collect metrics provided by clusters metrics databricks runtime 13.1 using bash script

Re: cluster metrics databricks runtime 13.1

User16539034020 — Thu, 11 Jan 2024 23:32:53 GMT

Hello,

Thanks for contacting Databricks.

To collect metrics from Databricks Runtime 13.1 clusters using a bash script, you generally need to interact with Databricks' API. We are contemplating the introduction of a cluster metrics API that could potentially fulfill this requirement. However, this feature is still in the planning stages, we're unable to provide an ETA yet.

Regards,
Monica

Re: cluster metrics databricks runtime 13.1

seefoods — Tue, 16 Jan 2024 16:11:09 GMT

Hello @User16539034020 ,
So how can i collect cluster metrics ganglia on a runtime databricks which is grater or equal than 13.1?
cordially,
Aubert

Re: cluster metrics databricks runtime 13.1

User16539034020 — Wed, 17 Jan 2024 16:41:34 GMT

hi, @Aubert:

Currently, you could only use static downloadable snapshots.
https://docs.databricks.com/en/compute/cluster-metrics.html

Regards,