Did you ever figure out what is causing the memory leak?  We are experiencing a nearly identical issue where the memory gradually increases over time and OOM after a few days.  

I did track down this open bug ticket that states there is a memory leak when a dataset is persisted even if it is unpersisted.

https://issues.apache.org/jira/browse/SPARK-35262