Unity Catalog: Metastore 3 level Hierarchy

Vasu_Kumar_T
Databricks Partner

I have data files categorized by application and region. Want to know the best way to load them into the Bronze and Silver layers while maintaining proper segregation.

For example, in our landing zone, we have a structure of raw files to be loaded using Autoloader with files from multiple applications and regions as in below. With a three-level naming convention in Unity catalog, I am concerned about difficulty in tracing table to file mapping in the future. Can you suggest the best possible architecture for this and explain its merits and internal organization? Thank you for your time and assistance.

Eg : Below is landing zone structure of raw files to be loaded using Autoloader.

        App1 : Region1 : File1

         App1 : Region1 : File2

        App2 : Region1 : File1

        App2 : Region1 : File2

There are multiple applications with around 11 regions and approx 100 files daily to be loaded