cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to delete strings from the /FileStore/

kkawka1
New Contributor III

We have just started working with databricks in one of my university modules, and the lecturers gave us a set of commands to practice saving data in the FileStore. One of the commands was the following:

dbutils .fs.cp("/ databricks - datasets / weathher / high_temps ", "/ FileStore /") 

This basically saved a bunch of strings in the FileStore without creating a new file. I have no idea how to remove them. Is this even possible?

6 REPLIES 6

Rishabh-Pandey
Esteemed Contributor

hey @Konrad Kawka​ you can use the command dbutils.fs.rm("your_filestore_path",True)

Rishabh Pandey

This will try to remove the /FileStore/ itself, which is not allowed

IllegalArgumentException: /FileStore is a system directory and cannot be deleted.

Anonymous
Not applicable

Hi @Konrad Kawka​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

kkawka1
New Contributor III

Hi @Vidula Khanna​ 

Unfortunately, I was not able to resolve the issue. I am beginning to think that it simply can't be done. After running this line of code:

dbutils .fs.cp("/databricks-datasets/weathher/high_temps ", "/ FileStore /") 

the contents of the high-temps.csv were saved in the FileStore itself as a long string. I don't know how to remove it. I've noticed that I can replace what is stored in the FileStore, for example by running this line of code:

dbutils .fs.cp("/databricks-datasets/flights/departuredelays.csv ", "/ FileStore /") 

Instead of adding more data, it replaces the contents of high_temps with the contents of departuredelays.csv. What I want to do, is to remove what I accidentally saved in the FileStore. I now know how to correctly save it in a new directory within the file store. I just want to remove what was saved in the root FileStore. I even managed to replace it all with an empty string. But what I would like to know is how to get access to the FileStore, see the list of subdirectories I created, and remove any data that I saved in the root FileStore by accident. Another issue is that when I try to run:

dbutils .fs.ls("/FileStore/")

It does not show me the list of files I created, and running this:

dbutils .fs.head("/FileStore/")

Returns a long string with the contents of one of the files I accidentally saved

kkawka1
New Contributor III

Hi @Vidula Khanna (Databricks)​ 

Unfortunately, I was not able to resolve the issue. I am beginning to think that it simply can't be done. After running this line of code:

dbutils .fs.cp("/databricks-datasets/weathher/high_temps ", "/ FileStore /") 

the contents of the high-temps.csv were saved in the FileStore itself as a long string. I don't know how to remove it. I've noticed that I can replace what is stored in the FileStore, for example by running this line of code:

dbutils .fs.cp("/databricks-datasets/flights/departuredelays.csv ", "/ FileStore /") 

Instead of adding more data, it replaces the contents of high_temps with the contents of departuredelays.csv. What I want to do, is to remove what I accidentally saved in the FileStore. I now know how to correctly save it in a new directory within the file store. I just want to remove what was saved in the root FileStore. I even managed to replace it all with an empty string. But what I would like to know is how to get access to the FileStore, see the list of subdirectories I created, and remove any data that I saved in the root FileStore by accident. Another issue is that when I try to run:

dbutils .fs.ls("/FileStore/")

It does not show me the list of files I created, and running this:

dbutils .fs.head("/FileStore/")

Returns a long string with the contents of one of the files I accidentally saved

Anonymous
Not applicable

Hi @Konrad Kawka​ 

 I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group