Thanks for the detailed explanation. Much Appreciate. As memory has 3 elements as below, can you suggest, given both #2 and #3 are part of JVM (on heap) memory, why do we need #3? when #3 is used over #2?

1. offheap
2. spark.executor.memory
3. spark.executor.memoryOverhead