Migrate delta live tables from a non-uc to uc enabled databricks workspace

[Fnazar]

Hello Team,

I have a usecase where I need to migrate Delta Live Tables from a non-Unity Catalog Databricks workspace to a Unity Catalog enabled Databricks workspace. What will be the efficient approach to achieve this scenario, please suggest?

[szymon_dybczak]

Hi @Fnazar ,

You can try to use UCX by Databricks Labs. It automates whole process. 

databrickslabs/ucx: Automated migrations to Unity Catalog (github.com)

[Fnazar]

Thank you for suggesting UCX by Databricks Labs. Could you please share any manual approaches for migrating Delta Live Tables from a non-Unity Catalog workspace to a Unity Catalog-enabled workspace?

I am looking for specific steps or best practices tailored to Delta Live Tables exclusively, such as exporting metadata, remapping schemas, or manually reconfiguring pipelines in the new workspace. Would appreciate any guidance on handling this migration manually.

[szymon_dybczak · Accepted solution]

Unfortunately, I can't share any opinions because I didn't have to migrate from hive to UC on my project. I suggested UCX because colleage of mine had good experience with migration using that tool.
But, since existing pipelines that use the Hive metastore cannot be upgraded to use Unity Catalog and to migrate an existing pipeline that writes to Hive metastore, you must create a new pipeline and re-ingest data from the data source(s) I would follow best practices that one of creators of DLT framework - Michael Armbrust mentions in below video:
Delta Live Tables in Depth Best Practices for Intelligent Data Pipelines (youtube.com)