topic Re: Unclear validation error message when adding a Unity Catalog metastore in Azure in Data Governance

Unclear validation error message when adding a Unity Catalog metastore in Azure

alexlod — Tue, 07 Feb 2023 21:20:16 GMT

I'm using Azure Databricks.

I've been following this guide to create a Unity Catalog metastore using a managed identity.

When I create the catalog and the validation step kicks off, I get this screen:

What does this mean?

If I click "Force create" and then assign the metastore to a workspace, I'm not able to write anything to the metastore. Eg, doing a simple "CREATE TABLE main.default.test" yields the following error:

[RequestId=<uuid> ErrorClass=INVALID_STATE] Failed to access cloud storage: AbfsRestOperationException exceptionTraceId=<uuid>

I believe that the failed validation is correct given I can't create a table, but I have no idea what's wrong. Any thoughts or pointers?

EDIT: formatting.

UPDATE: When creating the metastore, the "ADLS Gen 2 path" must exist in the storage container. Meaning, if the ADLS Gen 2 path value is "foo@bar.dfs.core.windows.net/baz", the "baz" folder must exist in the "foo" container. That gets the validation to pass. However, creating a table still creates the same error mentioned above.

UPDATE2: The issue was fully resolved by @Tom Smith's comment in this thread, which recommends to follow the upgrade documentation.

Re: Unclear validation error message when adding a Unity Catalog metastore in Azure

alexlod — Thu, 09 Feb 2023 19:26:43 GMT

Hi @Debayan Mukherjee, unfortunately that other thread isn't helpful because it's about AWS and I'm using Azure. Any other ideas? Thanks!

Re: Unclear validation error message when adding a Unity Catalog metastore in Azure

alexlod — Thu, 09 Feb 2023 23:17:37 GMT

I found the solution here: https://community.databricks.com/s/question/0D58Y00009pvzofSAA/unity-catalogerror-creating-table-errorclassinvalidstate-failed-to-access-cloud-storage-abfsrestoperationexception

Re: Unclear validation error message when adding a Unity Catalog metastore in Azure

alexlod — Fri, 17 Feb 2023 20:34:24 GMT

Read my "UPDATE2" for the answer/solution.

Re: Unclear validation error message when adding a Unity Catalog metastore in Azure

Anonymous — Sat, 08 Apr 2023 07:36:09 GMT

Hi @Alex Loddengaard

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you.

Cheers!

Re: Unclear validation error message when adding a Unity Catalog metastore in Azure

Debayan — Thu, 09 Feb 2023 19:01:21 GMT

Hi, Please look at the similar thread and let us know if that helps.