cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Problems with Unity Catalog

LukaszJ
Contributor III

Hello.

I am using Azure Databricks since a few months and I want to add a Unity Catalog.

Unfortunately, I have some problems:

1. I am not able to create a Unity Catalog Cluster with settings:

- Policy: Unrestricted

- Multi Node with Single user access mode

- Runtime version is 11.3 LTS with Spark 3.3.0 and Scala 2.12

- Worker type is Standard_F4s (1-2 workers)

- spark.databricks.dataLineage.enabled true

- spark.databricks.delta.preview.enabled true

Do you know what did I do wrong?

2. I cannot create a new metastore (probably because I do not have special cluster).

I read the instruction and I do not have "Create Metastore" button (I am using Data Science & Engineering mode)

image 

Could you help me with my problems?

Best regards,

Łukasz

12 REPLIES 12

Pat
Honored Contributor III

Hi @Łukasz Jaremek​ ,

not sure where to start, the Unity Catalog cluster is the last thing you need to do.

To add/enable Unity Catalog in your workspace you need to create metastore.

Metastore is account level Object I would say. There are some requirements to create that metastore, i.e. storage and policies in azure. Then you need to create the metastore in the Databricks account and attach that metastore(storage adbfs://).

After you have metastore, you attach that metastore to the workspace. It makes your workspace Unity Catalog enabled - make sure do it in dev workspace, as once attached workspace to UC, stays UC attached.

Then the next steps are creating cluster (so far we didn't have to use the cluster, everything was azure and databricks api).

This explains well how to do it: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started

--

2. I cannot create a new metastore (probably because I do not have special cluster).

I read the instruction and I do not have "Create Metastore" button (I am using Data Science & Engineering mode)

You shoud create metastore in the Databricks account (it's called E2 I believe), not in the workspace.

LukaszJ
Contributor III

Hello @Pat Sienkiewicz​ , thanks you for your response!

Okay, I have a new Azure Storage Account, and what next?

I have mounted it on the databricks with dbutils.mount, but what means "attach"?

I do not have button "Create Metastore" in Data tab.

Best regards,

Łukasz

Pat
Honored Contributor III

Hi @Łukasz Jaremek​ ,

you don't have to mount this storage account.

When you create new storage account (to use as metastore root account) you need to attach/set it as metastore storage on the account level:

go to Manage Account:

image 

then switch to metastore tab,

on the right side click create metastore (aka unity catalog)

image 

here you need to specify which storage account should be used:

imagein the next tab it will ask you to attach workspace to unity catalog.

thanks,.

Pat.

Hmm it is interesting.

I have a different app design than you.

I am a databricks admin and I have premium plan.

In my case it looks like this:

image source_ https___www.youtube.com_watch_v=QU8uxFkox1c&ab_channel=AdvancingAnalyticsDo you know why?

Pat
Honored Contributor III

I see you are on workspace level now, do you have possibility in top right corner to switch to the account level ? You need admin account permission:

source; https://www.databricks.com/blog/2022/08/26/databricks-workspace-administration-best-practices-for-ac...

image

I have a Admin on Databricks.

At Azure level I have Contributor role. There are one "better" (Owner) but it is only for assign roles in Azure RBAC.

I have created a new Azure Databricks (again with premium plan and admin account) and still I have simple "User Settings" interface.

> "do you have possibility in top right corner to switch to the account level ?" - Do you mean change my databricks user for admin or something else?

Pat
Honored Contributor III

I mean that you need to click on „manage account”, as I have showed before. If you don’t have account level permission then you won’t see this option I think.

image 

you are currently in wrong „place”. To create unity catalog / metastore you need to sign i to databricks account, not the workspace.

thanks,

Pat.

Pat
Honored Contributor III

Okay I do not know what is wrong with my Databricks.

I clicked on my email, then "Manage Account",then I selected my Databricks instantion (the same as the photo from your link) and after that I am in the same Databricks workspace. There are no any difference 😞

User settings and Data page are the same.

Maybe there are some extra steps like creating or enabling something?

Based on the documentation, I have Azure Contributor, a premium plan and an admin account.

Could someone from the administration please look at my problem?

I don't know who to tag so I apologize for the inaccurate tag.

@Kaniz Fatma​ 

shudong
New Contributor II

I also cannot find the "Create Metastore" button from my console in Azure Databricks, although I am the one create my databricks stack... Could anyone help on this?

Addi1
New Contributor II

I'm facing the same issues as other customers listed above. I do not have the "Create Metastore" button available.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group