โ12-22-2023 03:50 PM
I'm the account admin for the subscription where the databricks workspace is created. However If I open the account console, its prompting me to select one of the databricks workspaces. Please note that its my own subscription via visual studio benefits in my company tenant.
โ12-23-2023 01:39 PM
This is ok to have one metastore. You have several ways to restrict access to specific catalog by ACL or Bind Catalog to specific Workspaces. Please read documentation about Unity best practice
Your organization can create like 3 catalogs for your project and you can bring 3 data lake storages dedicated for this project. The. You bind those catalogs to those Storages and to your Workspaces. Admin can give you ownership over catalog , so you can do wathever you want inside.
All under one Metastore so your organization can manage all proejects centralny and if there will be need share data between projects in easy and secure way.
You don't split environments to different metastores, you do this on catalog level.
You can , but you dont have to split environments per Storage Accouns, you can split it e.g. on Container level.
There are multiple architecture but I encaurge you to spend some time with docs.
https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html#organize-your-data
โ12-23-2023 05:28 AM
There is no such a role as Account Admin on subscription. You probably have Owner role over your subscription and that is not sufficient.
You need global tenant admin to login and grant you the Databricks account admin role.
โ12-23-2023 09:28 AM
Attaching the screenshot.
Does Databricks account admin have access to all the workspaces in my organization ? If it is, I may not get the access.
Do I need to ask my tenant admin to create an unity catalog metastore and assign it to my workspace ? Any instructions that I can share with the admin ?
โ12-23-2023 10:08 AM
Account Admin can link Workspace with Unity metastore, if it is linked, he can grant himself Admin privilege to this workspace.
So most likely you won't get it. You could try to create dev tenant and connect your subscription to dev tenant ( it is free) but I don't know if this will work with MSDN subscription.
You can share this instruction
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/create-metastore
But if there is existing metastore in your tenant , probably you should just ask to link your workspace to existing and get access to some kind of sandbox catalog, there are many ways but this is only put to your admins.
โ12-23-2023 11:17 AM
If there is an existing metastore for the region, doesn't mean my organization store all the meta data across the workspaces in a specific metastore or data lake ?
Databricks suggests to have one metastore per region. How the project specific metadata and access can be managed ? Moreover storing the project specific meta data in our own data lake and storing the project data in our own external lake would make more sense.
Having a metastore for all the workspaces (Dev, UAT, Prod) and external data lakes separately per environment is being followed across industry ?
โ12-23-2023 01:39 PM
This is ok to have one metastore. You have several ways to restrict access to specific catalog by ACL or Bind Catalog to specific Workspaces. Please read documentation about Unity best practice
Your organization can create like 3 catalogs for your project and you can bring 3 data lake storages dedicated for this project. The. You bind those catalogs to those Storages and to your Workspaces. Admin can give you ownership over catalog , so you can do wathever you want inside.
All under one Metastore so your organization can manage all proejects centralny and if there will be need share data between projects in easy and secure way.
You don't split environments to different metastores, you do this on catalog level.
You can , but you dont have to split environments per Storage Accouns, you can split it e.g. on Container level.
There are multiple architecture but I encaurge you to spend some time with docs.
https://docs.databricks.com/en/data-governance/unity-catalog/best-practices.html#organize-your-data
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group