Caching is for performance optimization but it may not work, if there is a problem lies in the logic of your window function, data skew, or data inconsistencies. 

I would recommend to try with a memory optimized cluster to see how it goes.