topic Re: delete files from the directory in Data Engineering

delete files from the directory

db-avengers2rul — Tue, 29 Nov 2022 13:28:48 GMT

Is there a way to delete recursively files using a command in notebooks

since in the below directory i have many combination of files like .txt,,png,.jpg but i only want to delete files with .csv

example dbfs:/FileStore/.csv*

db-avengers2rul — Tue, 29 Nov 2022 13:30:37 GMT

%fs rm -r dbfs:/FileStore/.csv*

dbutils.fs.rm('dbfs:/FileStore/.csv*', True)

is this correct ?

UmaMahesh1 — Sat, 03 Dec 2022 08:31:30 GMT

Hi @Rakesh Reddy Gopidi

You can use the os module to iterate over a directory.

By using a loop over the directory, you can check what the file ends with using .endsWith(".csv).

After fetching all the files, you can remove it.

Hope this helps..

Cheers.