cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Setting up Unity Catalog in Azure

leelee3000
New Contributor III
New Contributor III

Trying to create a metastore that will be connected to an external storage (ADLS) but we don't have the option to create a new metastore in 'Catalog' tab in the UI. Based on some research, we see that we'll have to go into "Manage Account" and then create a metastore from there. However, once we click "Manage Account" from the drop down menu under our username on the top right, we are given the choice of selecting the workspace. When we pick the correct workspace, we are directed back to the home page of the workspace instead of the account page. We are both owners of this workspace so I'm not sure what other permissions we need to grant ourselves to solve this. The workspace is also a premium workspace.

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz_Fatma
Community Manager
Community Manager

Hi @leelee3000, Creating a metastore in Unity Catalog and linking it to workspaces involves a few steps.

 

Let’s break it down:

 

Create a Storage Location for Metastore-Level Storage:

  • In your Azure account, set up a storage location specifically for managing tables and volumes at the metastore level.

Create an Azure Managed Identity:

  • Next, create an Azure-managed identity that will have access to the storage location you just set up.

Create the Metastore:

  • In Azure Databricks, create the metastore and attach it to the storage location.
  • Assign workspaces to this metastore.

Here are the detailed steps:

 

Step 1: Create a Storage Container for Metastore-Level Managed Storage:

  • This container will hold metadata about securable objects (such as tables, volumes, external locations, and shares) and the permissions governing access to them.
  • Remember that each metastore exposes a three-level namespace (catalog.schema.table) for organizing data.

Step 2: Create an Azure Managed Identity:

  • This identity will grant access to the storage location.
  • Ensure that you meet the following requirements:
    • You must be an Azure Databricks account admin.
    • The first Azure Databricks account admin must be a Microsoft Enterprise ID (formerly Azure Active Directory Global Administrator) when they first log in to the Azure Databricks account console.
    • Upon first login, that user becomes an Azure Databricks account admin and no longer needs the Microsoft Enterprise ID Global Administrator role to access the Azure Databricks account.
    • The first account admin can assign users in the Microsoft Enterprise ID tenant as additional account admins (who can themselves assign more account admins). Additional account admins do not require specific roles in Microsoft Enterprise ID.
    • The workspaces you attach to the metastore must be on the Azure Databricks Premium plan.

Step 3: Create the Metastore and Attach Workspaces:

  • In Azure Databricks, create the metastore, attach the storage location, and assign workspaces to the metastore.

Remember that if your workspace was automatically enabled for Unity Catalog when it was created, a metastore is automatically provisioned for your workspace. You only need to perform these tasks if you were prompted to do so during workspace creation.

 

Feel free to follow these steps, and you’ll have your metastore set up for seamless data management! 🌟

View solution in original post

4 REPLIES 4

Kaniz_Fatma
Community Manager
Community Manager

Hi @leelee3000, Creating a metastore in Unity Catalog and linking it to workspaces involves a few steps.

 

Let’s break it down:

 

Create a Storage Location for Metastore-Level Storage:

  • In your Azure account, set up a storage location specifically for managing tables and volumes at the metastore level.

Create an Azure Managed Identity:

  • Next, create an Azure-managed identity that will have access to the storage location you just set up.

Create the Metastore:

  • In Azure Databricks, create the metastore and attach it to the storage location.
  • Assign workspaces to this metastore.

Here are the detailed steps:

 

Step 1: Create a Storage Container for Metastore-Level Managed Storage:

  • This container will hold metadata about securable objects (such as tables, volumes, external locations, and shares) and the permissions governing access to them.
  • Remember that each metastore exposes a three-level namespace (catalog.schema.table) for organizing data.

Step 2: Create an Azure Managed Identity:

  • This identity will grant access to the storage location.
  • Ensure that you meet the following requirements:
    • You must be an Azure Databricks account admin.
    • The first Azure Databricks account admin must be a Microsoft Enterprise ID (formerly Azure Active Directory Global Administrator) when they first log in to the Azure Databricks account console.
    • Upon first login, that user becomes an Azure Databricks account admin and no longer needs the Microsoft Enterprise ID Global Administrator role to access the Azure Databricks account.
    • The first account admin can assign users in the Microsoft Enterprise ID tenant as additional account admins (who can themselves assign more account admins). Additional account admins do not require specific roles in Microsoft Enterprise ID.
    • The workspaces you attach to the metastore must be on the Azure Databricks Premium plan.

Step 3: Create the Metastore and Attach Workspaces:

  • In Azure Databricks, create the metastore, attach the storage location, and assign workspaces to the metastore.

Remember that if your workspace was automatically enabled for Unity Catalog when it was created, a metastore is automatically provisioned for your workspace. You only need to perform these tasks if you were prompted to do so during workspace creation.

 

Feel free to follow these steps, and you’ll have your metastore set up for seamless data management! 🌟

leelee3000
New Contributor III
New Contributor III

@Kaniz_Fatma Thank you so much for the quick response, you rock!

SSundaram
Contributor

Looks like you do not have account admin access.  

bsadler
New Contributor II

I have been wrestling with this question for days now.  I seem to be the only one with this question so I am sure I am doing something wrong.  I am trying to create a UC metastore but there is not an option in "Catalog" to create a metastore.  This seems to be leelee3000's quesiton.  I have read the Databricks documentation thoroughly and it reads a lot like kaniz_fatma's response.  This response is almost exactly what the documentation says.  In step it says, " In Azure Databricks, create the metastore, attach the storage location, and assign workspaces to the metastore."   The question is "how do I create a metastore when it is not available in the catalog" and the answer is "In Azure Databricks, create the metastore."  At this point, I would pay quite a bit to figure out how to enable my workspace with the UC, be able to create a compute that is UC enabled (I have done this Spark Init in the advanced section but the blue "Unity Catalog" icon does not appear in the Summary), and then finally create a UC Catalog so that I can create a volume to that I can implement and deploy a RAG model.   Again, very likely my user error, can someone help? 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!