topic Best practice for organizing data objects using Unity Catalog in Data Governance

Best practice for organizing data objects using Unity Catalog

ossinova — Fri, 17 Feb 2023 15:46:21 GMT

I was wondering if there is a best practice or recommended way to organize data objects (tables) in Unity Catalog.

Following the medallion architecture, how should one organize and name these three namespaces:

Catalog(s)
Schema(s)
Table(s)

Example:

Catalog: environment specific (dev / test / prod)
Schema: layer specific (bronze / silver / gold)
Table: tablename (sourceX_employee)

Example2:

Catalog: environment and source specific (dev_sourceX / test_sourceX / prod_sourceX)
Schema: layer specific (bronze / silver / gold)
Table: tablename (employee)

How are you organizing your tables?

Re: Best practice for organizing data objects using Unity Catalog

Anonymous — Thu, 23 Feb 2023 05:04:32 GMT

Hi @Oscar Dyremyhr

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Best practice for organizing data objects using Unity Catalog

jose_gonzalez — Thu, 23 Feb 2023 18:14:07 GMT

HI @Oscar Dyremyhr ,

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

Re: Best practice for organizing data objects using Unity Catalog

Avvar2022 — Fri, 24 Mar 2023 14:24:45 GMT

In Real time do we have 1 meta store for PROD and NON-PROD ? if Yes in the backend there would be a one storage account (ADLS Gen2) > Container for all Environments.