Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

190809 — Fri, 02 Dec 2022 16:38:33 GMT

I have just created a metastore to allow unity catalog to be enabled for my workspace. However I am getting the following error when trying to create a table as per the unity catalog demo notbook:

Error in SQL statement: UnauthorizedAccessException: PERMISSION_DENIED: Access denied. Cause: 403 Forbidden error from cloud storage provider. exceptionTraceId=f5101e43-1f10-4e5f-8c15-0be034d7af8a

A couple of questions.

If I delete the metastore will all of the data stored in my original hive-metastore be deleted ie. what existed before I created a metastore to enable unity catalog and assigned this to my existing workspace.
Is there a way to re-configure the metastore to check that the credentials are correct?

Thanks in advance.

Re: Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

Sivaprasad1 — Fri, 02 Dec 2022 20:15:32 GMT

Hi,

Please hit metastore list and storage credentials API or CLI to get the detail of metastore and Storage credentials and you can verify the config.

https://api-docs.databricks.com/rest/latest/unity-catalog-api-specification-2-1.html

https://docs.databricks.com/dev-tools/cli/unity-catalog-cli.html#metastores

You can reconfigure your metastore or update the existing credentials:

To update the metastore with a new storage credential, you can use the following CLI commands.

AWS: https://docs.databricks.com/dev-tools/cli/unity-catalog-cli

Azure: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/unity-catalog-cli

databricks --profile test-shard unity-catalog storage-credentials create \

--json '{"name": "test-role-cred", "aws_iam_role": {"role_arn": "arn:aws:iam:::role/us-east-1-extdev-managed-catalog-test-role"}}'

databricks --profile test-shard unity-catalog metastores update \ --id bd5c8b05-xyzw-476a-8b4e-08320f9633d2 \

--json '{"storage_root_credential_id":"3783164d-9887-4b1c-afc6-a0b06cf3f44c"}'

Once the metastore is updated, check the below configs with the highlighted parameters:

Check the storage credentials config

databricks unity-catalog list-storage-credentials

The output should look like this:

{

"storage_credentials": [

{

"id": "9e3e6c22-e765-4da7-xxxx-***",

"name": "4d173c3e-ee91-4c2f-a890-xxxxxxx-data-access-config-16607430xxxx",

"aws_iam_role": {

"role_arn": "arn:aws:I am:::role/my-unity-role",

"unity_catalog_iam_arn": "arn:aws:iam:::role/unity-catalog-prod-UCMasterRole-14S5ZJVKOWXYZ",

"external_id": "949b8e6e-f557-4aba-xxxx-***"

"owner": "kavya.parag+uc@databricks.com",

"metastore_id": "4d173c3e-ee91-4c2f-xxxx-***",

"created_at": 1660743064187,

"created_by": "kp+uc@databricks.com",

"updated_at": 1660743064187,

"updated_by": "kp+uc@databricks.com",

"used_for_managed_storage": true

}

]

}

Check the metastore config:

databricks unity-catalog list-metastores

The output should look like this:

{

"metastores": [

{

"name": "kp-test",

"storage_root": "s3://bucket",

"default_data_access_config_id": "9e3e6c22-e765-4da7-xxxx-***",

"storage_root_credential_id": "9e3e6c22-e765-4da7-xxxx-***",

"delta_sharing_scope": "INTERNAL",

"owner": "kp+uc@databricks.com",

"privilege_model_version": "0.1",

"metastore_id": "4d173c3e-ee91-4c2f-a890-***",

"created_at": 1660743063329,

"created_by": "kp+uc@databricks.com",

"updated_at": 1660743064838,

"updated_by": "kp+uc@databricks.com",

"cloud": "aws",

"region": "us-east-1",

"global_metastore_id": "aws:us-east-1:4d173c3e-ee91-4c2f-xxxx-***"

}

Re: Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

190809 — Mon, 05 Dec 2022 06:52:47 GMT

Thanks for your answer to the second part of my question, super helpful! Can you confirm the first part of my question as well please?

topic Re: Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook. in Data Governance

Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

Re: Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

Re: Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.