cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How I explain account vs workspace vs Unity Catalog metastore to new platform engineers

techinlayers
New Contributor II

Sharing the explanation that finally stopped the same question coming back every onboarding.

The account is the organisation-level container, normally one per cloud provider, and it owns billing, account admins and identities. A workspace is a single deployment of the UI with its own notebooks, clusters, jobs and dashboards, which is why dev, test and prod are usually separate workspaces. The Unity Catalog metastore is created in the account console: one per region, linked to any number of workspaces in that region, holding catalogs, schemas, tables, views and volumes plus the permissions on them.

The two details that clear up most confusion:
1. The metastore is regional and shared, not per workspace. That is what lets dev, test and prod see one catalog.schema.table namespace.
2. Identity federation means users, groups and service principals are configured once in the account console and assigned to workspaces, so you grant access to a group once.

Storage stays in your cloud account: metastore-level managed storage is optional, and existing data is registered as an external location with a storage credential.

I made a 2:32 animated diagram of it: https://youtu.be/2yy9-BL_RDo
The series in order: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4

Interested in how others structure catalogs when one regional metastore covers several environments.

Account, workspace, metastore: three Databricks terms that interviewers love to ask about together, because the relationship between them is where most people get confused. This video builds the hierarchy from the top down: one account for the organisation, dev, test and prod workspaces under it ...
0 REPLIES 0