cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to capture the heap dump of the Spark driver JVM

User16869510359
Esteemed Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

User16869510359
Esteemed Contributor

Find the DriverDaemon

%sh
jps

Take the heap dump

%sh
jmap -dump:live,format=b,file=pbs_worker_DriverDaemon.hprof 2413

Copy out to download

%sh
cp pbs_worker_DriverDaemon.hprof /dbfs/FileStore/pbs_worker_04-30-2021T15-50-00.hprof

View solution in original post

1 REPLY 1

User16869510359
Esteemed Contributor

Find the DriverDaemon

%sh
jps

Take the heap dump

%sh
jmap -dump:live,format=b,file=pbs_worker_DriverDaemon.hprof 2413

Copy out to download

%sh
cp pbs_worker_DriverDaemon.hprof /dbfs/FileStore/pbs_worker_04-30-2021T15-50-00.hprof

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.