How to capture the heap dump of the Spark driver JVM

[brickster_2018 · Accepted solution]

Find the DriverDaemon

%sh
jps

Take the heap dump

%sh
jmap -dump:live,format=b,file=pbs_worker_DriverDaemon.hprof 2413

Copy out to download

%sh
cp pbs_worker_DriverDaemon.hprof /dbfs/FileStore/pbs_worker_04-30-2021T15-50-00.hprof