Unity Catalog will allow you to bring your own HMS

[User16826994223]

Anyone know more about how the Unity Catalog will allow you to bring your own HMS (eg Glue)?

Will this be treated as a separate 'catalog', which you can access but you can't use the other features of Unity Catalog on eg ABAC etc

Any reading on this topic?

[Kaniz]

Hi @Kunal Gaurav​ ,

Unity Catalog (Preview)

Unity Catalog is in Public Preview. The account console UI for Unity Catalog is in Private Preview. To participate in the preview, contact your Databricks representative.

Unity Catalog is a fine-grained governance solution for data and AI on the Lakehouse.

Unity Catalog helps simplify security and governance of your data with the following key features:

• Define once, secure everywhere: Unity Catalog offers a single place to administer data access policies that apply across all workspaces and personas.
• Standards-compliant security model: Unity Catalog’s security model is based on standard ANSI SQL, and allows administrators to grant permissions at the level of catalogs, databases (also called schemas), tables, and views in their existing data lake using familiar syntax.
• Built-in auditing: Unity Catalog automatically captures user-level audit logs that record access to your data.

Note:-

Review the limitations for Unity Catalog during this public preview.

The account console is a web UI from which you can create and manage metastores, link metastores to workspaces, and manage account-level users and groups who can access data in metastores. Unity Catalog stores certain information in the account console.

Unity Catalog requires a Databricks account on the Premium plan.

In this guide:

• Get started using Unity Catalog
• Key concepts
• Data permissions
• Key tasks
• Create compute resources
• Create a metastore
• Create catalogs and schemas
• Manage account-level identities
• Create tables
• Create views
• Manage access to data
• Manage external locations and storage credentials
• Query data
• Train a machine-learning model with Python from data in Unity Catalog
• Connect to BI tools
• Audit access and activity for Unity Catalog resources
• Upgrade tables and views to Unity Catalog
• Automate Unity Catalog setup using Terraform
• Limitations

[zpappa]

@Kunal Gaurav​ yes, it is treated as a synthetic catalog. You can query it by using the convention "hive_metastore" as the catalog name. i.e. SELECT * FROM hive_metastore.schema_name.table_name

This will work for internal HMS, external HMS and Glue.

You cannot set any Unity ACLs on these HMS/Glue objects currently.

You can use the Upgrade Wizard in the DBSQL Explorer to upgrade HMS/Glue schemas and tables to Unity Catalog.