cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

ganglia is unavailable on DBR 13.0

HamidHamid_Mora
New Contributor II

We created a library in databricks to ingest ganglia metrics for all jobs in our delta tables;However end point 8652 is no more available on DBR 13.0

is there any other endpoint available ? since we need to log all metrics for all executed jobs not only for running clusters which is only visuel

or is there qny other way that I am not aware of

Thanks

2 REPLIES 2

Hubert-Dudek
Esteemed Contributor III

Ganglia is only supported on Databricks Runtime versions 12 and below. From Databricks Runtime 13, Ganglia is replaced by a new Databricks metrics system offering more features and integrations. To export metrics to external services, you can use Datadog or Azure Log Analytics integration as recommended best practices https://learn.microsoft.com/en-us/azure/databricks/clusters/clusters-manage

Thanks for your reply, however I dont see cluster metrics like cpu,memory and network in log analytics options on databricks