ganglia is unavailable on DBR 13.0

[HamidHamid_Mora]

We created a library in databricks to ingest ganglia metrics for all jobs in our delta tables;However end point 8652 is no more available on DBR 13.0

is there any other endpoint available ? since we need to log all metrics for all executed jobs not only for running clusters which is only visuel

or is there qny other way that I am not aware of

Thanks

[Hubert-Dudek]

Ganglia is only supported on Databricks Runtime versions 12 and below. From Databricks Runtime 13, Ganglia is replaced by a new Databricks metrics system offering more features and integrations. To export metrics to external services, you can use Datadog or Azure Log Analytics integration as recommended best practices https://learn.microsoft.com/en-us/azure/databricks/clusters/clusters-manage

[HamidHamid_Mora]

Thanks for your reply, however I dont see cluster metrics like cpu,memory and network in log analytics options on databricks