Unexpected Schema ID Folder Creation in Unity Catalog External Location

Sunil_Poluri
Databricks Partner

I've set up Unity Catalog with an external location pointing to a storage account. For each schema, I’ve configured a dedicated container path. For example:

abfss://schemas@<storage_account>.dfs.core.windows.net/_unityStorage/schemas/<schema_id>

When I create a schema, a schema_id is generated. I expect this schema_id to be reflected as a folder under the schema container path, like:

/_unityStorage/schemas/<schema_id>

However, I’ve noticed that this folder doesn’t appear immediately—presumably because no objects (like tables) exist yet.

Here’s what I’ve observed:

  • When I create a Delta table within the schema, I expect the table data to be stored under the schema’s storage path.
  • Similarly, when I create a DLT pipeline targeting the same schema, I expect the tables to be stored under the same schema path.
  • But instead, a new schema ID folder gets created in the storage account under the schema container—even though the schema name is the same.

My question is: Under what conditions does Unity Catalog generate a new schema_id folder in the storage account, even when the schema name hasn’t changed?

Any insights or documentation references would be greatly appreciated!