how do i delete files from the DBFS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-18-2017 04:48 AM
I can't see where in the databricks UI that I can delete files that have been either uploaded or saved to the DBFS - how do I do this?
- Labels:
-
DBFS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-03-2017 03:14 PM
you can ssh into your cluster if you configured it with a public key at creation time. BUT this question is still relevant because I am having trouble deleting files in the /dbfs directory. I accidentally ended up creating a huge number of streams with different chekcpoints paths in my top level dbfs directory. now when I try to
rm -rf checkpoint_path
, it takes FOREVER to delete. And I can't use my cluster since they consume so much memory. They wont even go away when I make an entirely new cluster!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-18-2019 04:04 PM
Open a notebook and run the command
dbutils.fs.rm("/FileStore/tables/your_table_name.csv")
referencing this link
https://docs.databricks.com/data/databricks-file-system.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2021 09:53 PM
This works! Thanks! To be abundantly clear, this means "Open a Python notebook in Databricks, be sure it's attached to an active cluster, and then run the below command"
dbutils.fs.rm("/FileStore/tables/<your_table_name>.csv")