I have a Unity Catalog Enabled cluster with Node type Standard_DS4_v2 (28 GB Memory, 8 Cores). When "Use Photon Acceleration" option is disabled spark.executor.memory is 18409m. But if I enable Photon Acceleration it shows spark.executor.memory as 4602m. Due to this most of the code which I have written is failing giving an error

org.apache.spark.memory.SparkOutOfMemoryError: Photon ran out of memory while executing this query.

Photon Enabled Cluster:

• Spark Version: 13.3.x-photon-scala2.12
• Executor Memory: 4602m

Photon Disabled Cluster:

• Spark Version: 13.3.x-scala2.12
• Executor Memory: 18409m

1. Why enabling photon reduces the executor memory?
2. Is there a way to keep spark.executor.memory same as 18409m with photon feature enabled?