Issue setting metastore in GCP Databricks

[manish05485]

While setting up metastore in GCP Databricks, I added the bucket name and then  service account permissons as well. Still my catalog dont have base root location. This deters me from creating table in my catalog. 

Root storage credential for metastore XXXXXX does not exist. Please contact your Databricks representative or consider updating the metastore with a valid storage credential https://docs.databricks.com/api- explorer/workspace/metastores/update

I followed this documentation, this root storage credential thing is nowhere mentioned in the documentation
https://docs.gcp.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html#enable-works...

[Kaniz]

Hi @manish05485., 

I understand that you’re facing an issue with setting up the metastore in GCP Databricks. The error message about the missing root storage credential can indeed be confusing.

Let’s break it down and find a solution.

1. Root Storage Credential:

   • The “root storage credential” is essential for managing external locations within the Unity Catalog. It allows Databricks to access storage resources (like buckets or containers) associated with your metastore.
   • Without this credential, Databricks cannot create tables or interact with storage locations effectively.

2. Creating the Root Storage Credential:

   • To resolve this issue, you’ll need to create the root storage credential. Here are the steps:

     a. Account Admin Privileges: - You must have account admin privileges to create storage credentials. - If you’re not an account admin, consider reaching out to someone who has these permissions.

     b. Create Storage Credentials: - Go to your Databricks workspace and navigate to the Unity Catalog. - Create the storage credentials associated with your storage location (bucket or container). - These credentials allow Databricks to authenticate and access the storage resources.

     c. Update the Metastore: - Use Postman or any REST API client to call the Databricks REST API. - Update the metastore with the unique ID of the storage credentials you just created. - Confirm that the storage credential ID is correctly associated with the metastore.

3. External Locations:

   • Additionally, consider setting up an external storage location within the Data Catalog.
   • This step might not be explicitly mentioned in the documentation, but it can help ensure proper configuration.
   • External locations scope access to specific storage paths.

4. Documentation Links:

   • Here are the relevant documentation links for your reference:
     • Manage External Locations and Storage Credentials
     • Create a Unity Catalog Metastore

I hope this helps you resolve the issue! If you have any further questions or need additional assistance, feel free to ask. 😊

[manish05485]

@Kaniz 
Why is this not in the following documentation. Cant this thing be entirely setup via UI without going to api routes, because business leaders prefer setting up things via UI forms etc , instead of technical nitty gritty. 
https://docs.gcp.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html#enable-works... 

Additionally is there step by step demo of this?

[Ayushi_Suthar]

Hi @manish05485 , Good Day! 

Error:Root storage credential for metastore XXXXXX does not exist. Please contact your Databricks representative or consider updating the metastore with a valid storage

Error states that the data access configuration for the metastore does not exist or don't have storage credential associated as well, for that we can list storage credentials and verify if the root storage credential exists we could associate that, if that doesn't exist please refer doc[1] to create one and that can be associated with the metastore. You could use Databricks CLI to perform these actions:

a. Get the storage credential Id by running the below command: 
>> databricks unity-catalog list-storage-credentials

Document to refer : https://docs.gcp.databricks.com/en/archive/dev-tools/cli/unity-catalog-cli.html#list-available-stora...

b. Create a file and save the storage credential as update-metastore.json
{
"storage_root_credential_id": "<storage-credential-id>"
}

c.Once the file has been created run the below command to associate the storage credential: 

databricks unity-catalog metastores update --id 12a345b6-7890-1cd2-3456-e789f0a12b34 \
                                           --json-file update-metastore.json

Please refer to doc[2] for more details on updating metastore settings.

Doc[1]: https://docs.gcp.databricks.com/en/connect/unity-catalog/index.html#how-does-unity-catalog-connect-o...

Doc[2] : https://docs.gcp.databricks.com/en/archive/dev-tools/cli/unity-catalog-cli.html#update-a-metastores-...

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi