topic Re: Unity Catalog hive_metastore schemas in Administration & Architecture

Unity Catalog hive_metastore schemas

CarolinaK — Wed, 23 Oct 2024 11:13:13 GMT

Hi all,

Apologies if this is the wrong group but I was looking in Unity Catalog and noticed that you have different schemas in the hive_metastore depending on if you select a cluster or if you select a warehouse. Could someone please explain what the differences are between schemas attached to a cluster and those attached to a warehouse? Information on how they're created, managed, accessed etc. as well as any pros or cons of each approach would be really appreciated! I've tried looking through the documentation but I haven't been able to find anything concrete and hope someone here might be able to enlighten me/point me in the right direction.

Thanks,

Carolina

Re: Unity Catalog hive_metastore schemas

Stefan-Koch — Wed, 23 Oct 2024 16:53:20 GMT

Hi Carolina

It's not quite clear what you mean? One thing is schemas that are in the hive_metastore, the other is schemas that are in the Unity Catalog. But it is not the case that schemas are explicitly assigned to a cluster. This means that there are differences between schemas that are in the hive metastore and those that are in the Unity Catalog.

Differences between Unity Catalog and Hive Metastore are listed here:
https://docs.databricks.com/en/data-governance/unity-catalog/hive-metastore.html

In general, you can say that you should no longer work with Hive Metastore, but only with Unity Catalog.

Re: Unity Catalog hive_metastore schemas

CarolinaK — Thu, 24 Oct 2024 12:48:29 GMT

Hi Stefan,

Apologies for not being clear! I'm quite new to Databricks so might not be able to describe exactly what I'm stuck on very well... Effectively, when I enter Unity Catalog, I can choose between the cluster or the SQL warehouse.

Say I select Cluster1, the view I see for example is:

> hive_metastore

> schema1

> schema2

> schema3

> schema4

But then if I select the SQL warehouse, I see for example:

> hive_metastore

> schema1

> schema3

So there are some schemas in the hive_metastore are in both the cluster and the SQL warehouse but some schemas exist in the cluster but not the SQL warehouse. I don't really understand why this is, what the differences are between the two, what this means when it comes to granting permissions or anything else (I'm a bit lost I'm afraid!).

Hope that makes more sense!

Re: Unity Catalog hive_metastore schemas

Stefan-Koch — Thu, 24 Oct 2024 12:56:56 GMT

Hi Carolina

Do you have Screenshots?

Re: Unity Catalog hive_metastore schemas

navallyemul — Sun, 27 Oct 2024 17:52:25 GMT

No schemas are directly attached to compute resources, whether it's an all-purpose cluster or a SQL warehouse in serverless mode.