Unity catalog Migration

[Phani1]

Hi Team,

Could you please help me to understand, 

 1)Why we need to migrate Unity catalog? if we are not migrating what benefits we will not get?

2) How to migrate Unity catalog (What all are objects needs to migrate and any tool) ? 

Regards,

Phanindra

[Kaniz]

Hi @Phani1, Good Question!

Why Migrate to Unity Catalog?

• Unity Catalog is a unified governance solution for Databricks workspaces. Without it, each Databricks workspace connects to a Hive metastore and maintains a separate service for Table Access Controls (TACL).
• The need for migration arises due to the following reasons:
  • Consistency: Unity Catalog ensures consistent metadata across workspaces, avoiding manual synchronization issues.
  • Data and Access Control: It centralizes views, table definitions, and ACLs, simplifying management and enhancing security.
  • Partition Management: Unity Catalog manages partitions differently than Hive, offering improved partition handling.
  • Streamlined Management: Upgrading to Unity Catalog streamlines table and view management.
• In summary, migrating to the Unity Catalog provides better governance, consistency, and ease of mana....

How to Migrate to Unity Catalog?

• To migrate tables and views from your existing Hive metastore to Unity Catalog, follow these steps:
  • Requirements:
    • A storage credential with information about a service principal authorized to access the table's location path.
    • An external location referencing your cloud tenant's storage credential and data path.
    • CREATE EXTERNAL TABLE permission on the external locations of the tables to be upgraded.
  • Upgrade Process:
    1. Open the Catalog Explorer in Databricks.
    2. Select the hive_metastore as your catalog and choose the schema (database) you want to upgrade.
    3. Click Upgrade at the top right of the schema detail view.
    4. Select the external tables you wish to upgrade (only formats supported by Unity Catalog can be upgraded).
    5. Set the destination catalog, schema, and owner for each table.
    6. Review table configurations and modify them if needed.
    7. Click Create Query for Upgrade to generate SQL statements.
    8. Run the query to copy each table's metadata from Hive metastore to Unity Catalog.
    9. The upgraded tables are marked as such in the upgrade wizard.
  • Fine-Grained Access Control:
    • Define permissions for each new table using the Permissions tab.

Remember that enabling Unity Catalog will prevent users from accessing data in the metastore, so pla....

Happy migrating! 🚀