cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

Cannot delete file in dbfs

Go to solution
erigaud
Honored Contributor

โ€Ž08-11-2023 06:58 AM

Hello, 

I am trying to delete a folder in /dbfs/mnt, but I am unable to do so. The folder was an old mounted storage account, which was deleted. The folder contains a single file 'mount.err'.

In command line I tried rm -rf my_folder, with and without sudo. I also tried to rename the folder. 

Whatever I try, I get the following error : rm: cannot remove 'my_folder/mount.err': Input/output error

I also tried in a notebook and with dbutils, but one again without success.

I tried repairing the filesystem with fsck.ext4, without success.

Please let me know if someone has a solution, thank you.

0 Kudos
1 ACCEPTED SOLUTION

Accepted Solutions

Go to solution
Priyanka_Biswas
Databricks Employee
Databricks Employee

โ€Ž08-23-2023 03:26 PM

Hi @erigaud  The issue you're facing might be due to the fact that the folder was a mounted storage account that has been deleted. This could cause some inconsistencies in the file system view and hence, you're unable to delete the folder.
You can try using Databricks Utilities to refresh the mounts and then attempt to delete the folder again.
Here is the command you can use in Databricks notebook:

python
dbutils.fs.refreshMounts()
dbutils.fs.rm("/mnt/my_folder", recurse=True)

This command will refresh the state of the mounts in your workspace and then attempt to delete the folder recursively.

View solution in original post

2 REPLIES 2

Go to solution
erigaud
Honored Contributor
In response to Priyanka_Biswas

โ€Ž08-23-2023 11:11 PM

Hello,

Thanks for the answer !

The problem was indeed coming from a broken mount point. After running the refreshMounts() command, I still was not able to remove the folder, because the command was throwing a NullPointerException. However I was able to run the dbutils.fs.unmount command on the broken folder (even though before it did not work and the folder was not showing up when I was trying to list all the mounts.

After running the unmount command I was able to delete the folder just fine. Thank you !

0 Kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group
Announcements