Hello @sowanth!

Off-heap memory is automatically configured on some clusters to improve stability and reduce Java garbage collection issues, particularly for Photon or heavy caching workloads. This setting isn’t coming from your repo or policies but is applied at the cluster level. If your Spark jobs don’t require this much off-heap memory, you can adjust it by overriding spark.memory.offHeap.enabled and spark.memory.offHeap.size in the cluster’s Spark configuration.

https://kb.databricks.com/en_US/clusters/spark-executor-memory