cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

delete non empty folders from workspace

Cloudtek
New Contributor

Looking for a solution to delete 1000's of non-empty folders that I created wrongly while migration. (looking for solution that delete thoudands of folders created with below name)

Folders have a path like "shared/db/..../..../..../yippy.dbc"

1 REPLY 1

Kaniz_Fatma
Community Manager
Community Manager

Hi @CloudtekTo delete thousands of non-empty folders in Databricks, you can use the following approaches:

  1. Databricks Notebook Command:

  2. Parallel Deletion with Apache Spark™:

    • If you have a large number of folders to delete, consider using an Apache Spark™ job to list and delete files faster. Here’s how:
  3. Databricks CLI:

Remember to replace /mnt/my_folder and dbfs:/foobar with the actual paths of the folders you need to delete. Be cautious when deleting data to avoid accidental loss. 🚀

 
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!