I set up an Azure Data Lake Storage (ADLS) account with containers named metastore, bronze, silver, gold, and source. I created a Unity Catalog metastore in Databricks via the admin console, and I created a container called metastore in my Data Lake. I defined external locations for each container (e.g., abfss://bronze@<storage_account>.dfs.core.windows.net/) and created a catalog without specifying a location, assuming it would use the metastore's default location. I also created schemas (bronze, silver, gold) and assigned each schema to the corresponding container's external location (e.g., bronze schema mapped to the bronze container).

In my source container, I have a folder structure: customers/customers.csv.

I built a Delta Live Tables (DLT) pipeline with the following configuration:

-- Bronze table

CREATE OR REFRESH STREAMING TABLE my_catalog.bronze.customers

AS

SELECT *, current_timestamp() AS ingest_ts, _metadata.file_name AS source_file

FROM STREAM read_files(

'abfss://source@<storage_account>.dfs.core.windows.net/customers',

format => 'csv'

);

-- Silver table

CREATE OR REFRESH STREAMING TABLE my_catalog.silver.customers

AS

SELECT *, current_timestamp() AS process_ts

FROM STREAM my_catalog.bronze.customers

WHERE email IS NOT NULL;

-- Gold materialized view

CREATE OR REFRESH MATERIALIZED VIEW my_catalog.gold.customers

AS

SELECT count(*) AS total_customers

FROM my_catalog.silver.customers

GROUP BY country;

• Why are my tables stored under this unity/schemas/<schema_id>/tables/<table_id> structure instead of directly in customers/parquet_files with a _delta_log folder in the respective containers?

• How can I configure my DLT pipeline or Unity Catalog setup to ensure the tables are stored in the bronze, silver, and gold containers with a folder structure like customers/parquet_files and _delta_log?

• In industry-level projects, how do teams typically manage table storage locations and folder structures in ADLS when using Unity Catalog and Delta Live Tables? Are there best practices or common configurations to ensure a clean, predictable folder structure for bronze, silver, and gold layers?