@Retired_mod thanks for the reply.

I'm going to try it but I don't think it fully addresses the issue. According to your explanation, given a 8GB worker, on default, it will reserve ~800MB for the overhead memory. It still leaves ~7GB available, and yet, the platform limited  spark.executor.memory to around 2GB (see screenshot in initial post). There could be other reserves for other things, but it is decent portion of memory left that should be available that I should be able to get allocation for.

EDIT: I tried lowering spark.executor.memoryOverheadFactor. The limitation value is still the same.