cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

delete non empty folders from workspace

Cloudtek
New Contributor

Looking for a solution to delete 1000's of non-empty folders that I created wrongly while migration. (looking for solution that delete thoudands of folders created with below name)

Folders have a path like "shared/db/..../..../..../yippy.dbc"

1 REPLY 1

Kaniz_Fatma
Community Manager
Community Manager

Hi @CloudtekTo delete thousands of non-empty folders in Databricks, you can use the following approaches:

  1. Databricks Notebook Command:

  2. Parallel Deletion with Apache Sparkโ„ข:

    • If you have a large number of folders to delete, consider using an Apache Sparkโ„ข job to list and delete files faster. Hereโ€™s how:
  3. Databricks CLI:

Remember to replace /mnt/my_folder and dbfs:/foobar with the actual paths of the folders you need to delete. Be cautious when deleting data to avoid accidental loss. ๐Ÿš€

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group