-werners-
Esteemed Contributor III

Databricks does not use Yarn AFAIK (see this topic).

The memory allocation is handled by spark.executor.memory.

The amount of memory available for each executor is allocated within the Java Virtual Machine (JVM) memory heap.

Here is some more detail:

Azure

AWS

You can also do a test run on a cluster and then monitor the workers and driver using Ganglia, which gives you a view on what's goin on and how much memory is allocated/used.