topic Re: how do i delete files from the DBFS in Data Engineering

how do i delete files from the DBFS

tthorpe — Thu, 18 May 2017 11:48:03 GMT

I can't see where in the databricks UI that I can delete files that have been either uploaded or saved to the DBFS - how do I do this?

Re: how do i delete files from the DBFS

PaulLintilhac — Tue, 03 Oct 2017 22:14:58 GMT

you can ssh into your cluster if you configured it with a public key at creation time. BUT this question is still relevant because I am having trouble deleting files in the /dbfs directory. I accidentally ended up creating a huge number of streams with different chekcpoints paths in my top level dbfs directory. now when I try to

rm -rf checkpoint_path

, it takes FOREVER to delete. And I can't use my cluster since they consume so much memory. They wont even go away when I make an entirely new cluster!

Re: how do i delete files from the DBFS

SophieGou — Tue, 19 Nov 2019 00:04:14 GMT

Open a notebook and run the command

dbutils.fs.rm("/FileStore/tables/your_table_name.csv")

referencing this link

https://docs.databricks.com/data/databricks-file-system.html

Re: how do i delete files from the DBFS

agonsa — Sat, 03 Jul 2021 04:53:04 GMT

This works! Thanks! To be abundantly clear, this means "Open a Python notebook in Databricks, be sure it's attached to an active cluster, and then run the below command"

dbutils.fs.rm("/FileStore/tables/<your_table_name>.csv")