Databricks Community

CarolinaK · ‎10-23-2024

Hi all,

Apologies if this is the wrong group but I was looking in Unity Catalog and noticed that you have different schemas in the hive_metastore depending on if you select a cluster or if you select a warehouse. Could someone please explain what the differences are between schemas attached to a cluster and those attached to a warehouse? Information on how they're created, managed, accessed etc. as well as any pros or cons of each approach would be really appreciated! I've tried looking through the documentation but I haven't been able to find anything concrete and hope someone here might be able to enlighten me/point me in the right direction.

Thanks,

Carolina

Stefan-Koch · ‎10-23-2024

Hi Carolina

It's not quite clear what you mean? One thing is schemas that are in the hive_metastore, the other is schemas that are in the Unity Catalog. But it is not the case that schemas are explicitly assigned to a cluster. This means that there are differences between schemas that are in the hive metastore and those that are in the Unity Catalog.

Differences between Unity Catalog and Hive Metastore are listed here:
https://docs.databricks.com/en/data-governance/unity-catalog/hive-metastore.html

In general, you can say that you should no longer work with Hive Metastore, but only with Unity Catalog.

CarolinaK · ‎10-24-2024

Hi Stefan,

Apologies for not being clear! I'm quite new to Databricks so might not be able to describe exactly what I'm stuck on very well... Effectively, when I enter Unity Catalog, I can choose between the cluster or the SQL warehouse.

Say I select Cluster1, the view I see for example is:

> hive_metastore

> schema1

> schema2

> schema3

> schema4

But then if I select the SQL warehouse, I see for example:

> hive_metastore

> schema1

> schema3

So there are some schemas in the hive_metastore are in both the cluster and the SQL warehouse but some schemas exist in the cluster but not the SQL warehouse. I don't really understand why this is, what the differences are between the two, what this means when it comes to granting permissions or anything else (I'm a bit lost I'm afraid!).

Hope that makes more sense!