error creating catalog in Unity Catalog – EXTERNAL_LOCATION_DOES_NOT_EXIST and admin console storage

[chexa_Wee]

Hi all, I’m trying to create a new catalog in Azure Databricks Unity Catalog but I’m running into issues. When I tried to add a default path in the Admin Console → Metastore settings, I got this error: “Metastore storage root URL does not exist. Please provide a storage location for the catalog (for example 'CREATE CATALOG myCatalog MANAGED LOCATION <location-path>'). Alternatively set up a metastore root storage location.” I’m on Azure Databricks Premium (free trial) with Unity Catalog enabled. My storage account is STORAGE123 and the container is warehousetest, and I already set up a Data Connector for this storage. Do I need to explicitly create a Storage Credential and External Location in Unity Catalog even though the container is already connected with a Data Connector? Is the metastore-level storage path mandatory, or can catalogs be created only with external locations? What’s the correct setup order (Admin console path → external location → catalog)? Any guidance or step-by-step instructions would be really appreciated.

Thanks!

[ManojkMohan]

Metastore Root Storage (Managed Location)

• This is a mandatory storage path if you want to create managed catalogs.
• It’s essentially the “default warehouse” location where Unity Catalog can create tables, databases, and managed data.
• Path must exist in your Azure Data Lake Storage Gen2 container. Simply having a Data Connector is not enough.

Proposed solution try the below

Create a Storage Credential

I would recommend use service principal as it enables RBAC-based access.

Set Metastore Root Storage (Mandatory if you want Managed Catalogs)

• In Admin Console → Metastore → Settings → Storage Root,

Create the Catalog

Once the storage root or an external location is ready:

• Managed catalog (uses metastore root):

If you skip the location and the metastore root isn’t set, you’ll get the error you saw.

[szymon_dybczak]

Hi @chexa_Wee ,

Starting from November 9, 2023, Databricks by default won't configure metastore-level storage for managed tables and volumes. 
Databricks recommends that you create a separate managed storage location for each catalog in your metastore.

So, when you want to create new catalog you just need to provide a path for all your managed objects.Managed storage is a dedicated storage location in your Azure account for managed tables and managed volumes.

Assigning managed storage to a catalog requires that you create:

• A storage credential.
• An external location that references that storage credential.

Here, you have detailed documenation how to create catalog using UI and SQL with managed storage:

Create catalogs - Azure Databricks | Microsoft Learn

[chexa_Wee]

Hello @szymon_dybczak ,

thank you for the reply.

Could you please provide some clarification? After logging into the admin console and selecting the metastore, I chose 'Set metastore-level storage' to update the ADLS Gen2 Path. I provided the path to an externally created storage account: abfss://containername@storageaccountname.dfs.core.windows.net/.

When I do this, I get an error. Should I be using the path to the default storage account that was created with the Databricks workspace?

I'd really appreciate the help.

[szymon_dybczak]

Hi @chexa_Wee ,

So once again. If I were you I would stick to what Databricks recommends and I would just configured managed storage location at the Catalog level (not at metastore level as you're trying now). 
Anyway, if you still want to configure it at existing metastore then just follow this guide. The steps are clearly defined in documentation, so you should be able to make it work:

Manage Unity Catalog metastores - Azure Databricks | Microsoft Learn

[chexa_Wee]

Hi @szymon_dybczak ,
I actually tried creating it like this:
CREATE CATALOG IF NOT EXISTS rnd_projects
MANAGED LOCATION 'abfss://<ContainerName>@<StorageAccount>.dfs.core.windows.net/';
But I got the following error:
ErrorClass=EXTERNAL_LOCATION_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST] External location path does not exist.
I also connected with Azure Access Connector for databricks with access to storage account giving azure bob data contributor.

[szymon_dybczak]

But did you even follow the guide? You need to first create storage location and external location.

To create storage location you just need to replicate steps mentioned below:

Create a storage credential for connecting to Azure Data Lake Storage - Azure Databricks | Microsoft...

Then create external location using storage credentials created at above step. Again, use below guide to create external location:

Create an external location to connect cloud storage to Azure Databricks - Azure Databricks | Micros...

Once you will have both storage credentials and external location then you should be able to create Unity Catalog with managed location without issues.

[chexa_Wee]

Thank you @szymon_dybczak 
I will try this.

[szymon_dybczak]

Great, let us know how it went