10-23-2024 04:10 AM - edited 10-23-2024 04:13 AM
Hi all,
Apologies if this is the wrong group but I was looking in Unity Catalog and noticed that you have different schemas in the hive_metastore depending on if you select a cluster or if you select a warehouse. Could someone please explain what the differences are between schemas attached to a cluster and those attached to a warehouse? Information on how they're created, managed, accessed etc. as well as any pros or cons of each approach would be really appreciated! I've tried looking through the documentation but I haven't been able to find anything concrete and hope someone here might be able to enlighten me/point me in the right direction.
Thanks,
Carolina
10-23-2024 09:53 AM
Hi Carolina
It's not quite clear what you mean? One thing is schemas that are in the hive_metastore, the other is schemas that are in the Unity Catalog. But it is not the case that schemas are explicitly assigned to a cluster. This means that there are differences between schemas that are in the hive metastore and those that are in the Unity Catalog.
Differences between Unity Catalog and Hive Metastore are listed here:
https://docs.databricks.com/en/data-governance/unity-catalog/hive-metastore.html
In general, you can say that you should no longer work with Hive Metastore, but only with Unity Catalog.
10-24-2024 05:48 AM
Hi Stefan,
Apologies for not being clear! I'm quite new to Databricks so might not be able to describe exactly what I'm stuck on very well... Effectively, when I enter Unity Catalog, I can choose between the cluster or the SQL warehouse.
Say I select Cluster1, the view I see for example is:
> hive_metastore
> schema1
> schema2
> schema3
> schema4
But then if I select the SQL warehouse, I see for example:
> hive_metastore
> schema1
> schema3
So there are some schemas in the hive_metastore are in both the cluster and the SQL warehouse but some schemas exist in the cluster but not the SQL warehouse. I don't really understand why this is, what the differences are between the two, what this means when it comes to granting permissions or anything else (I'm a bit lost I'm afraid!).
Hope that makes more sense!
10-24-2024 05:56 AM
Hi Carolina
Do you have Screenshots?
10-27-2024 10:52 AM
No schemas are directly attached to compute resources, whether it's an all-purpose cluster or a SQL warehouse in serverless mode.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group