cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Move a delta table from a non UC metastore to a UC metastore preserving history

Jotav93
New Contributor II

Hi,

 

I am using Azure databricks and we recently enabled UC in our workspace. We have some tables in our non UC metastore that we want to move to a UC enabled metastore. Is there any way we can move these tables without loosing the delta table history? (i.e. sql command "describe history old_mestastore.db_name.table_name" should give exactly the same results than "describe history new_metstore.db_name.table_name" where old metastore is a non UC metastore while new_metastore is a UC enabled metastore).

 

Any info would be appreciated.

 

Thanks

Joaquim

2 REPLIES 2

ThomazRossito
New Contributor III

Hello,
It is possible to have the expected result with dbutils.fs.cp("Origin location", "Destination location", True) and then create the table with the LOCATION of the Destination location

Hope this helps

Att.
Thomaz Antonio Rossito Neto
Data Technical Lead / Data Engineer-Architect

Hi Thomasz,

thanks for your answer. It partially works as it preserves the versions but it doesn't preserve version timestamps (I didn't specify that explicitly but I need to preserve version timestamps).

- Version before the copy

Jotav93_0-1713338360117.png

- Copy operation

Jotav93_1-1713338390241.png

Table creation and version after the copy

Jotav93_3-1713338535304.png

Am I doing anything wrong? Would you suggest either move or .zip and unzip to preserve file modified times? Should I create the table first and then copy the data inside? Or what would you suggest?

Thanks

 

Joaquim