topic Re: How to delete strings from the /FileStore/ in Data Engineering

How to delete strings from the /FileStore/

kkawka1 — Thu, 16 Feb 2023 16:42:28 GMT

We have just started working with databricks in one of my university modules, and the lecturers gave us a set of commands to practice saving data in the FileStore. One of the commands was the following:

dbutils .fs.cp("/ databricks - datasets / weathher / high_temps ", "/ FileStore /")

This basically saved a bunch of strings in the FileStore without creating a new file. I have no idea how to remove them. Is this even possible?

Re: How to delete strings from the /FileStore/

Rishabh-Pandey — Thu, 16 Feb 2023 18:19:45 GMT

hey @Konrad Kawka you can use the command dbutils.fs.rm("your_filestore_path",True)

Re: How to delete strings from the /FileStore/

kkawka1 — Thu, 16 Feb 2023 18:23:22 GMT

This will try to remove the /FileStore/ itself, which is not allowed

IllegalArgumentException: /FileStore is a system directory and cannot be deleted.

Re: How to delete strings from the /FileStore/

Anonymous — Fri, 17 Feb 2023 05:04:36 GMT

Hi @Konrad Kawka

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: How to delete strings from the /FileStore/

kkawka1 — Fri, 17 Feb 2023 08:58:59 GMT

Hi @Vidula Khanna

Unfortunately, I was not able to resolve the issue. I am beginning to think that it simply can't be done. After running this line of code:

dbutils .fs.cp("/databricks-datasets/weathher/high_temps ", "/ FileStore /")

the contents of the high-temps.csv were saved in the FileStore itself as a long string. I don't know how to remove it. I've noticed that I can replace what is stored in the FileStore, for example by running this line of code:

dbutils .fs.cp("/databricks-datasets/flights/departuredelays.csv ", "/ FileStore /")

Instead of adding more data, it replaces the contents of high_temps with the contents of departuredelays.csv. What I want to do, is to remove what I accidentally saved in the FileStore. I now know how to correctly save it in a new directory within the file store. I just want to remove what was saved in the root FileStore. I even managed to replace it all with an empty string. But what I would like to know is how to get access to the FileStore, see the list of subdirectories I created, and remove any data that I saved in the root FileStore by accident. Another issue is that when I try to run:

dbutils .fs.ls("/FileStore/")

It does not show me the list of files I created, and running this:

dbutils .fs.head("/FileStore/")

Returns a long string with the contents of one of the files I accidentally saved

Re: How to delete strings from the /FileStore/

kkawka1 — Fri, 24 Feb 2023 10:03:17 GMT

Hi @Vidula Khanna (Databricks)

Unfortunately, I was not able to resolve the issue. I am beginning to think that it simply can't be done. After running this line of code:

dbutils .fs.cp("/databricks-datasets/weathher/high_temps ", "/ FileStore /")

dbutils .fs.cp("/databricks-datasets/flights/departuredelays.csv ", "/ FileStore /")

dbutils .fs.ls("/FileStore/")

It does not show me the list of files I created, and running this:

dbutils .fs.head("/FileStore/")

Returns a long string with the contents of one of the files I accidentally saved

Re: How to delete strings from the /FileStore/

Anonymous — Sat, 11 Mar 2023 03:25:40 GMT

Hi @Konrad Kawka

I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.