https://community.databricks.com/t5/data-engineering/do-ganglia-report-incorrect-memory-stats/m-p/20752#M14046 <P>Ganglia reports the memory utilization at the system level. </P><P>Say for example if the JVM has Xmx value of 100 GB. At some point, it will occupy 100GB and then with a Garbage collection, it will clear off the heap. Once the GC frees up the memory, the memory is not given back to the customer. That means at the system level the JVM process is still utilizing 100GB. Although the heap usage is much less than that. </P> Thu, 24 Jun 2021 15:18:10 GMT brickster_2018 2021-06-24T15:18:10Z https://community.databricks.com/t5/data-engineering/do-ganglia-report-incorrect-memory-stats/m-p/20751#M14045 <P>I am looking at the memory utilization of the executors and I see the heap utilization of the executor is far less than what is reported in the Ganglia. Why do ganglia report incorrect memory details.</P> Thu, 24 Jun 2021 14:51:39 GMT https://community.databricks.com/t5/data-engineering/do-ganglia-report-incorrect-memory-stats/m-p/20751#M14045 brickster_2018 2021-06-24T14:51:39Z https://community.databricks.com/t5/data-engineering/do-ganglia-report-incorrect-memory-stats/m-p/20752#M14046 <P>Ganglia reports the memory utilization at the system level. </P><P>Say for example if the JVM has Xmx value of 100 GB. At some point, it will occupy 100GB and then with a Garbage collection, it will clear off the heap. Once the GC frees up the memory, the memory is not given back to the customer. That means at the system level the JVM process is still utilizing 100GB. Although the heap usage is much less than that. </P> Thu, 24 Jun 2021 15:18:10 GMT https://community.databricks.com/t5/data-engineering/do-ganglia-report-incorrect-memory-stats/m-p/20752#M14046 brickster_2018 2021-06-24T15:18:10Z