topic Upgrade HSM to UC using Ucx tool - workspace to workspace migration in Data Engineering

Upgrade HSM to UC using Ucx tool - workspace to workspace migration

ShivMukesh — Tue, 20 Aug 2024 07:37:34 GMT

Hello team,

I understand that an automatic upgrade to UC utilizing the UCx tool (Databricks Lab project) is now available to complete this migration from HSM to UC in automate way. But does this tool allow workspace to workspace catalog/artifact migration ? That is, I have an HSM workspace, and if the client wants to upgrade the Catalog to UC with a new or another (fresh) workspace rather than an in-place migration. So, does the UCX tool provide this capability for workspace to workspace migrations ? If not, please let me know of an alternative strategy to automating this procedure using UCx or any other Lab initiatives. Kindly assist me if any limitations around those approaches as well.

Re: Upgrade HSM to UC using Ucx tool - workspace to workspace migration

mark_ott — Mon, 17 Nov 2025 11:37:08 GMT

The UCx (Databricks Labs UCx tool) automates many aspects of migrating from HSM (Hive Metastore) to Unity Catalog (UC), but its default operation focuses on upgrading the catalog within the same workspace (an in-place migration). As of the most recent updates, UCx does not natively support automatic migration of catalogs/artifacts across different workspaces—that is, it does not directly handle a full workspace-to-workspace migration to a fresh workspace with Unity Catalog.

UCx Tool Capabilities and Limitations

The UCx tool is designed primarily for upgrading the catalog and artifacts within the original source workspace.
It automates catalog conversion, permissions, and data lineage mostly for the in-place upgrade scenario.
Artifacts such as tables, managed/external locations, ACLs, and lineage are usually mapped within the same workspace context.
Direct workspace-to-workspace automation—the kind you described—is not provided out of the box by UCx as of late 2025.

Alternative Migration Strategies

If you need to migrate HSM artifacts to UC in a new or different workspace, you can consider the following approaches:

Manual Export/Import: Use Databricks’ export utilities or custom scripts to export tables, views, and permissions from the HSM workspace, then reconstruct them in the new workspace with UC enabled.
Delta Lake Data: If tables are backed by Delta Lake, you can "register" the underlying Delta tables in a new workspace/catalog after copying physical files.
Workspace API Automation: Develop automation scripts using Databricks REST API to recreate catalogs, databases, tables, and permissions in the target workspace after upgrading to UC.
Databricks Partner Solutions/Labs Projects: Watch for new Databricks Labs projects or partner tools, but as of now there’s no officially supported or widely used tool for workspace-to-workspace automated UC migration.

Limitations to Consider

Identity & Permissions: Mapping and migrating ACLs between workspaces is complex, as identity (users/groups) might differ.
Physical Data Location: Migrating managed tables might require physically moving files in cloud storage; UCx does not automate this if the storage buckets differ between workspaces.
Lineage/History Loss: Full lineage and usage history might not transfer automatically between workspaces.
Manual Effort: Even script-based automation will require careful validation and may not be fully hands-off.

Summary Table

Approach	Automation Level	Supported by UCx	Limitations
In-place UCx migration	High	Yes	Only same workspace
Workspace-to-workspace w/ UCx	Low	No	Manual scripting needed
Delta Lake register in new WS	Medium	No	Data path must be accessible
REST API/scripted recreation	Medium	No	Identity/permissions challenges
Manual export/import	Low	No	Labor-intensive

In conclusion, the UCx tool does not currently support fully automated workspace-to-workspace migrations. You will need to employ alternative methods such as API-driven or scripted automation, manual procedures, or leverage Delta Lake features as appropriate, keeping in mind limitations regarding identities, permissions, and lineage transfer in multi-workspace migration scenarios.

Re: Upgrade HSM to UC using Ucx tool - workspace to workspace migration

Raman_Unifeye — Mon, 17 Nov 2025 15:31:20 GMT

I was involved in the simialr request in past where we did use UCX for the in-place workpsace. However, as we had to split few data assets to a different workspace, it was pretty much done using custom workarounds as mentioned above.

Re: Upgrade HSM to UC using Ucx tool - workspace to workspace migration

nayan_wylde — Mon, 17 Nov 2025 15:49:57 GMT

@ShivMukesh I have used UCX to migrate to Unity catalog. It is a great tool. But it also involves lot of workarounds specially in group migration and table migration. In group migration it renames the old workspace group and assigns the same permission to the account group. You have to run some custom scripts to cleanup the old group. Also it didn't suppported view migration while I was working starting this year.