Hi Suteja,

Thanks for the quick reply.

I have already tried the ```spark.catalog.clearCache()``` method but it doesn't work and was actually the reason for me posting the question.

In your code example, you are calling unperist on the dataframe after we have cleared the cache. Just for my information, why would we call unpersist on the dataframe if we have already cleared the cache of the session (assuming it would work).

For clarity, ```df.unpersist()``` does work, but this is cumbersome to implement in my application as the df is created in a local scope and is referred to by other scopes. I want to unpersist the Df only at the end of my application, where I do not have access to the Df variable anymore. Therefore I simply want to clear the cache of the whole spark cluster at the end of my application.

On the last part: I am calling the spark.catalog.clearCache() on the same spark session in which I persist my data. The spark context and sql context are also derived from that same sparksession.