topic Re: Unity catalog: separate workspace or not? in Data Governance

Unity catalog: separate workspace or not?

-werners- — Mon, 29 Aug 2022 08:45:05 GMT

Now that Unity Catalog is GA, I was wondering how y'all implemented it (or planning to do so) on a workspace level.

Do you create a separate 'one workspace to rule them all' with the Unity Catalog, or do you create it on an already existing workspace (PRD f.e.)?

Re: Unity catalog: separate workspace or not?

Abishek — Mon, 29 Aug 2022 08:51:39 GMT

Hello @Werner Stinckens Can you please refer to the below doc for UC

https://docs.databricks.com/data-governance/unity-catalog/get-started.html

Re: Unity catalog: separate workspace or not?

Prabakar — Mon, 29 Aug 2022 08:54:47 GMT

I believe it will depend on the use case.

Re: Unity catalog: separate workspace or not?

Pat — Mon, 29 Aug 2022 11:53:07 GMT

Are you thinking about creating totally separate workspace just to manage Unity Catalog?

I've been playing a bit with the Unity Catalog and I am currently thinking about managing it from one of the existing workspaces. I don't think there is need for separate one, unless you have some spare once.

I am thinking about DEV and PROD Unit Catalogs and DEV one will be managed from the existing DEV workspace and PROD from the existing PROD workspace.

On one hand it sounds tempting if I got you correctly - similar to having the AWS account just for CI/CD.

Re: Unity catalog: separate workspace or not?

-werners- — Mon, 29 Aug 2022 11:55:42 GMT

Well that is one of the options. I know it is not a technical requirement but it could be interesting.

On the other hand: We already have a workspace so why create another one...

Hence my question 🙂

Re: Unity catalog: separate workspace or not?

Pat — Mon, 29 Aug 2022 12:06:24 GMT

I didn't think about it before, but it sounds interesting to have another workspace with limited access to do the administration work.

In my case I will go rather with the DEV/TEST (ws) -> PROD (ws) that we use only to develop and execute ETL pipelines.

Re: Unity catalog: separate workspace or not?

prasadvaze — Sun, 25 Dec 2022 03:17:54 GMT

@Pat Sienkiewicz There can be only 1 Unity Catalog metastore per azure region and if your DEV and PROD are in one region then it means all our dev and prod catalogs will be in single metastore. This also means as long as you have admin permission on databricks account and metastore , you can login to any dev or prod workspace to manage the metasore content (such as storage credentials, external locations, catalogs / schemas , grp/user permissions etc ).

@Prabakar Ammeappin and @Abishek Subramanian I wish the databricks account admin console UI had features to manage storage credentials, external locations, catalogs / schemas , grp/user permissions so that we don't have to perform metastore admin tasks from a workspace UI.