topic Clear Cache From a Notebook, not from a Cluster in Data Engineering

Clear Cache From a Notebook, not from a Cluster

iptkrisna — Wed, 14 Jun 2023 10:04:40 GMT

Hi, I'm running all my jobs on one big cluster, I'm just concerned is there a solution on how we could clear cache resulted by a notebook in the end of the job when its done? hence it does not causing any memory problem sometime from one to another, e.g a job cannot running anymore due to the cache is already too big from other jobs.

Thank you.

Re: Clear Cache From a Notebook, not from a Cluster

Anonymous — Fri, 16 Jun 2023 07:21:47 GMT

Hi @krisna math

We haven't heard from you since the last response from @Debayan Mukherjee , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Re: Clear Cache From a Notebook, not from a Cluster

Debayan — Thu, 15 Jun 2023 07:29:47 GMT

Hi, You can try this thread https://community.databricks.com/s/question/0D53f00001okbqdCAA/how-to-clear-all-cache-without-restarting-the-cluster , let me know if this helps.

Also, please tag @Debayan with your next comment so I will be getting notified.