cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Any way to access unity catalog location through python/dbutils

ashish577
New Contributor III

I have a table created at unity catalog that was dropped, the files are not deleted due to the 30 day soft delete. Is there anyway to copy the files to a different location? When I try to use dbutils.fs.cp I get location overlap error with unity catalog managed location.

3 REPLIES 3

Atanu
Databricks Employee
Databricks Employee

I think you can not do that, as this are already into soft delete. 

what is the actual error you got?

Tharun-Kumar
Databricks Employee
Databricks Employee

@ashish577 

You can restore the file using your cloud service provider. Then you can create a table on the top of the restored files.

youssefmrini
Databricks Employee
Databricks Employee

 

You can use the dbutils.fs.mv command to move the files from the deleted table to a new location. Here's an example of how to do it:

 

python
# Define the paths
source_path = "dbfs:/mnt/<unity-catalog-location>/<database-name>/<table-name>"
target_path = "dbfs:/mnt/<new-location>/<path>"

# Move the files using dbutils.fs.mv command
dbutils.fs.mv(source_path, target_path)
 

Replace <unity-catalog-location> with the name of the Unity Catalog location where the table was created, <database-name> with the name of the database containing the table, <table-name> with the name of the dropped table, <new-location> with the new location where you want to move the files, and <path> with any additional path elements needed for the target location.

By using dbutils.fs.mv, you can move the files from the deleted table to a new location without having to copy the files. Please note that moving files from a managed location of Unity Catalog is not recommended and it is against the best practice of using a Catalog. You should only use the Catalog interfaces for performing operations on managed tables.

I hope this helps!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group