Purge utility for databricks

[vijaykumar99535]

I have one requirement where I need to delete older data from a folder, and folder data can be partitioned or without partition. What approach to follow to build this utility?

[Kaniz]

Hi @vijay kumar​ , Databricks workspace administrators can use the admin settings to manage their workspace’s settings. The articles in this section explain the purpose and function of notable workspace settings.

Check this:-

• Purge workspace storage

[Anonymous]

Hi @vijay kumar​ 

We haven't heard from you since the last response from @Kaniz Fatma​ , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others. 

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

[vijaykumar99535]

Hi @Vidula Khanna​ @Kaniz Fatma​ 

It does not answer my real question. We mount adls path in databricks where we read and write the data. Suppose I have one partitioned (partitioned on audit) folder like below. I want to build an utility to delete the 3 days older partition like (/mnt/data/contact/audit=20230620). What is the best approach we can follow to do this. So that it also not impact the parent folder/other partitions.

/mnt/data/contact/audit=20230620

/mnt/data/contact/audit=20230621

/mnt/data/contact/audit=20230622

/mnt/data/contact/audit=20230623