cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Upgrade HSM to UC using Ucx tool - workspace to workspace migration

ShivMukesh
New Contributor

Hello team,

I understand that an automatic upgrade to UC utilizing the UCx tool (Databricks Lab project) is now available to complete this migration from HSM to UC in automate way. But does this tool allow workspace to workspace catalog/artifact migration ? That is, I have an HSM workspace, and if the client wants to upgrade the Catalog to UC with a new or another (fresh) workspace rather than an in-place migration. So, does the UCX tool provide this capability for workspace to workspace migrations ? If not, please let me know of an alternative strategy to automating this procedure using UCx or any other Lab initiatives. Kindly assist me if any limitations around those approaches as well.

3 REPLIES 3

mark_ott
Databricks Employee
Databricks Employee

The UCx (Databricks Labs UCx tool) automates many aspects of migrating from HSM (Hive Metastore) to Unity Catalog (UC), but its default operation focuses on upgrading the catalog within the same workspace (an in-place migration). As of the most recent updates, UCx does not natively support automatic migration of catalogs/artifacts across different workspacesโ€”that is, it does not directly handle a full workspace-to-workspace migration to a fresh workspace with Unity Catalog.

UCx Tool Capabilities and Limitations

  • The UCx tool is designed primarily for upgrading the catalog and artifacts within the original source workspace.

  • It automates catalog conversion, permissions, and data lineage mostly for the in-place upgrade scenario.

  • Artifacts such as tables, managed/external locations, ACLs, and lineage are usually mapped within the same workspace context.

  • Direct workspace-to-workspace automationโ€”the kind you describedโ€”is not provided out of the box by UCx as of late 2025.

Alternative Migration Strategies

If you need to migrate HSM artifacts to UC in a new or different workspace, you can consider the following approaches:

  • Manual Export/Import: Use Databricksโ€™ export utilities or custom scripts to export tables, views, and permissions from the HSM workspace, then reconstruct them in the new workspace with UC enabled.

  • Delta Lake Data: If tables are backed by Delta Lake, you can "register" the underlying Delta tables in a new workspace/catalog after copying physical files.

  • Workspace API Automation: Develop automation scripts using Databricks REST API to recreate catalogs, databases, tables, and permissions in the target workspace after upgrading to UC.

  • Databricks Partner Solutions/Labs Projects: Watch for new Databricks Labs projects or partner tools, but as of now thereโ€™s no officially supported or widely used tool for workspace-to-workspace automated UC migration.

Limitations to Consider

  • Identity & Permissions: Mapping and migrating ACLs between workspaces is complex, as identity (users/groups) might differ.

  • Physical Data Location: Migrating managed tables might require physically moving files in cloud storage; UCx does not automate this if the storage buckets differ between workspaces.

  • Lineage/History Loss: Full lineage and usage history might not transfer automatically between workspaces.

  • Manual Effort: Even script-based automation will require careful validation and may not be fully hands-off.

Summary Table

Approach Automation Level Supported by UCx Limitations
In-place UCx migration High Yes Only same workspace
Workspace-to-workspace w/ UCx Low No Manual scripting needed
Delta Lake register in new WS Medium No Data path must be accessible
REST API/scripted recreation Medium No Identity/permissions challenges
Manual export/import Low No Labor-intensive
 
 

In conclusion, the UCx tool does not currently support fully automated workspace-to-workspace migrations. You will need to employ alternative methods such as API-driven or scripted automation, manual procedures, or leverage Delta Lake features as appropriate, keeping in mind limitations regarding identities, permissions, and lineage transfer in multi-workspace migration scenarios.

Raman_Unifeye
Contributor III

I was involved in the simialr request in past where we did use UCX for the in-place workpsace. However, as we had to split few data assets to a different workspace, it was pretty much done using custom workarounds as mentioned above.


RG #Driving Business Outcomes with Data Intelligence

nayan_wylde
Esteemed Contributor

@ShivMukesh I have used UCX to migrate to Unity catalog. It is a great tool. But it also involves lot of workarounds specially in group migration and table migration. In group migration it renames the old workspace group and assigns the same permission to the account group. You have to run some custom scripts to cleanup the old group. Also it didn't suppported view migration while I was working starting this year.