cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

how do i delete files from the DBFS

tthorpe
New Contributor

I can't see where in the databricks UI that I can delete files that have been either uploaded or saved to the DBFS - how do I do this?

3 REPLIES 3

PaulLintilhac
New Contributor III

you can ssh into your cluster if you configured it with a public key at creation time. BUT this question is still relevant because I am having trouble deleting files in the /dbfs directory. I accidentally ended up creating a huge number of streams with different chekcpoints paths in my top level dbfs directory. now when I try to

rm -rf checkpoint_path
, it takes FOREVER to delete. And I can't use my cluster since they consume so much memory. They wont even go away when I make an entirely new cluster!

SophieGou
New Contributor II

Open a notebook and run the command

dbutils.fs.rm("/FileStore/tables/your_table_name.csv")

referencing this link

https://docs.databricks.com/data/databricks-file-system.html

agonsa
New Contributor II

This works! Thanks! To be abundantly clear, this means "Open a Python notebook in Databricks, be sure it's attached to an active cluster, and then run the below command"

dbutils.fs.rm("/FileStore/tables/<your_table_name>.csv")

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group