cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Clear Cache From a Notebook, not from a Cluster

iptkrisna
New Contributor III

Hi, I'm running all my jobs on one big cluster, I'm just concerned is there a solution on how we could clear cache resulted by a notebook in the end of the job when its done? hence it does not causing any memory problem sometime from one to another, e.g a job cannot running anymore due to the cache is already too big from other jobs.

Thank you.

2 REPLIES 2

Debayan
Databricks Employee
Databricks Employee

Hi, You can try this thread https://community.databricks.com/s/question/0D53f00001okbqdCAA/how-to-clear-all-cache-without-restar... , let me know if this helps.

Also, please tag @Debayanโ€‹ with your next comment so I will be getting notified.

Anonymous
Not applicable

Hi @krisna mathโ€‹ 

We haven't heard from you since the last response from @Debayan Mukherjeeโ€‹ โ€‹, and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others. 

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group