Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-16-2024 07:25 AM
@dmart - Let's take a step back..
If the file path you want to delete is as follows
/mnt/path/table
and the internal partitioned structure is
/mnt/path/table/date=2022-01-03
/mnt/path/table/date=2022-01-04
Sample code to recursively list and delete the same
def recursive_ls(path):
result = dbutils.fs.ls(path)
files = [file for file in result if file.isFile()]
directories = [dir for dir in result if dir.isDir()]
for dir in directories:
files += recursive_ls(dir.path)
return files
files = recursive_ls("<parent directory path>")
for file in files:
print(file.path)
<dbutils.fs.rm(file.path)>
If the two tables listed are delta table, could you please try checking if any properties are set within the table by checking describe extended related to deleted file retention etc.