topic Re: Unity Catalog metastore is down error in Data Engineering

Unity Catalog metastore is down error

alesventus — Tue, 25 Jul 2023 18:11:50 GMT

When I want to run notebook in databricks all queries, saves and read take really long and I found error message in the clusters event log that says: Metastore is down. So, I think cluster is not able to connect to the metastore right now. Could be this the reason? I tried to create another cluster but with the same result. Any help on this? Unity catalog`s Metastore is on ADLS gen2. and we use premium workspace.

After few minutes the command is finished. But simple load of delta table with few records takes 5 minutes or so.

Thanks

Re: Unity Catalog metastore is down error

alesventus — Thu, 27 Jul 2023 06:34:42 GMT

Hi Kaniz. Thanks for reply. Just want to add some information. Im not sure if this error message is related to legacy Hive metastore or to Unity Catalog. We use UC from the beginning. Our metastore is on external storage - see screenshot.

This error occurs in the clusters event log exactly 6 minutes after the start of the cluster. Every time we start the cluster that error is there.

Re: Unity Catalog metastore is down error

karthik_p — Fri, 28 Jul 2023 16:43:34 GMT

@alesventus what is size of your data and what type of cluster size you are using. also is your workspace region and metastore region same

Re: Unity Catalog metastore is down error

Giri-Patcham — Tue, 02 Jul 2024 15:42:26 GMT

@alesventus I think "Metastore is down" here is related to the legacy Hive metastore. You can find the stack trace in the driver logs.
In your case do you need Hivemetastore?

Re: Unity Catalog metastore is down error

alesventus — Thu, 04 Jul 2024 07:04:26 GMT

This issue is solely related to the VNET. Azure engineer must set up connection within VNET correctly.