11-15-2022 04:21 AM
Hello.
I am using Azure Databricks since a few months and I want to add a Unity Catalog.
Unfortunately, I have some problems:
1. I am not able to create a Unity Catalog Cluster with settings:
- Policy: Unrestricted
- Multi Node with Single user access mode
- Runtime version is 11.3 LTS with Spark 3.3.0 and Scala 2.12
- Worker type is Standard_F4s (1-2 workers)
- spark.databricks.dataLineage.enabled true
- spark.databricks.delta.preview.enabled true
Do you know what did I do wrong?
2. I cannot create a new metastore (probably because I do not have special cluster).
I read the instruction and I do not have "Create Metastore" button (I am using Data Science & Engineering mode)
Could you help me with my problems?
Best regards,
Łukasz
11-15-2022 07:04 AM
Hi @Łukasz Jaremek ,
not sure where to start, the Unity Catalog cluster is the last thing you need to do.
To add/enable Unity Catalog in your workspace you need to create metastore.
Metastore is account level Object I would say. There are some requirements to create that metastore, i.e. storage and policies in azure. Then you need to create the metastore in the Databricks account and attach that metastore(storage adbfs://).
After you have metastore, you attach that metastore to the workspace. It makes your workspace Unity Catalog enabled - make sure do it in dev workspace, as once attached workspace to UC, stays UC attached.
Then the next steps are creating cluster (so far we didn't have to use the cluster, everything was azure and databricks api).
This explains well how to do it: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started
--
2. I cannot create a new metastore (probably because I do not have special cluster).
I read the instruction and I do not have "Create Metastore" button (I am using Data Science & Engineering mode)
You shoud create metastore in the Databricks account (it's called E2 I believe), not in the workspace.
11-17-2022 12:57 AM
Hello @Pat Sienkiewicz , thanks you for your response!
Okay, I have a new Azure Storage Account, and what next?
I have mounted it on the databricks with dbutils.mount, but what means "attach"?
I do not have button "Create Metastore" in Data tab.
Best regards,
Łukasz
11-17-2022 01:15 AM
Hi @Łukasz Jaremek ,
you don't have to mount this storage account.
When you create new storage account (to use as metastore root account) you need to attach/set it as metastore storage on the account level:
go to Manage Account:
then switch to metastore tab,
on the right side click create metastore (aka unity catalog)
here you need to specify which storage account should be used:
in the next tab it will ask you to attach workspace to unity catalog.
thanks,.
Pat.
11-17-2022 01:51 AM
11-17-2022 01:55 AM
I see you are on workspace level now, do you have possibility in top right corner to switch to the account level ? You need admin account permission:
11-17-2022 11:39 AM
I have a Admin on Databricks.
At Azure level I have Contributor role. There are one "better" (Owner) but it is only for assign roles in Azure RBAC.
I have created a new Azure Databricks (again with premium plan and admin account) and still I have simple "User Settings" interface.
> "do you have possibility in top right corner to switch to the account level ?" - Do you mean change my databricks user for admin or something else?
11-17-2022 11:44 AM
I mean that you need to click on „manage account”, as I have showed before. If you don’t have account level permission then you won’t see this option I think.
you are currently in wrong „place”. To create unity catalog / metastore you need to sign i to databricks account, not the workspace.
thanks,
Pat.
11-17-2022 12:10 PM
@Łukasz Jaremek
more details ;
https://learn.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/
11-17-2022 03:02 PM
Okay I do not know what is wrong with my Databricks.
I clicked on my email, then "Manage Account",then I selected my Databricks instantion (the same as the photo from your link) and after that I am in the same Databricks workspace. There are no any difference 😞
User settings and Data page are the same.
Maybe there are some extra steps like creating or enabling something?
Based on the documentation, I have Azure Contributor, a premium plan and an admin account.
11-20-2022 11:51 PM
Could someone from the administration please look at my problem?
I don't know who to tag so I apologize for the inaccurate tag.
@Kaniz Fatma
11-20-2022 11:52 PM
Hi @Łukasz Jaremek, No problem!
Sure, We're looking forward to resolving your problem at the earliest.
11-21-2022 02:27 AM
I also cannot find the "Create Metastore" button from my console in Azure Databricks, although I am the one create my databricks stack... Could anyone help on this?
11-26-2022 09:46 AM
I'm facing the same issues as other customers listed above. I do not have the "Create Metastore" button available.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group