Any way to access unity catalog location through python/dbutils

ashish577
New Contributor III

I have a table created at unity catalog that was dropped, the files are not deleted due to the 30 day soft delete. Is there anyway to copy the files to a different location? When I try to use dbutils.fs.cp I get location overlap error with unity catalog managed location.

Atanu
Databricks Employee
Databricks Employee

I think you can not do that, as this are already into soft delete. 

what is the actual error you got?

Tharun-Kumar
Databricks Employee
Databricks Employee

@ashish577 

You can restore the file using your cloud service provider. Then you can create a table on the top of the restored files.

youssefmrini
Databricks Employee
Databricks Employee

 

You can use the dbutils.fs.mv command to move the files from the deleted table to a new location. Here's an example of how to do it:

 

python
# Define the paths
source_path = "dbfs:/mnt/<unity-catalog-location>/<database-name>/<table-name>"
target_path = "dbfs:/mnt/<new-location>/<path>"

# Move the files using dbutils.fs.mv command
dbutils.fs.mv(source_path, target_path)
 

Replace <unity-catalog-location> with the name of the Unity Catalog location where the table was created, <database-name> with the name of the database containing the table, <table-name> with the name of the dropped table, <new-location> with the new location where you want to move the files, and <path> with any additional path elements needed for the target location.

By using dbutils.fs.mv, you can move the files from the deleted table to a new location without having to copy the files. Please note that moving files from a managed location of Unity Catalog is not recommended and it is against the best practice of using a Catalog. You should only use the Catalog interfaces for performing operations on managed tables.

I hope this helps!