shan_chandra
Databricks Employee
Databricks Employee

@dmart - Let's take a step back.. 

If the file path you want to delete is as follows 

/mnt/path/table

and the internal partitioned structure is 

/mnt/path/table/date=2022-01-03
/mnt/path/table/date=2022-01-04

Sample code to recursively list and delete the same

def recursive_ls(path):
    result = dbutils.fs.ls(path)
    files = [file for file in result if file.isFile()]
    directories = [dir for dir in result if dir.isDir()]
    for dir in directories:
        files += recursive_ls(dir.path)
    return files

files = recursive_ls("<parent directory path>")
for file in files:
    print(file.path)
    <dbutils.fs.rm(file.path)>

If the two tables listed are delta table, could you please try checking if any properties are set within the table by checking describe extended related to deleted file retention etc.