Unity Catalog metastore is down error

[alesventus]

When I want to run notebook in databricks all queries, saves and read take really long and I found error message in the clusters event log that says: Metastore is down. So, I think cluster is not able to connect to the metastore right now. Could be this the reason? I tried to create another cluster but with the same result. Any help on this? Unity catalog`s Metastore is on ADLS gen2. and we use premium workspace.

After few minutes the command is finished. But simple load of delta table with few records takes 5 minutes or so.

Thanks

[Kaniz_Fatma]

Hi @alesventus, Yes, the cluster may becannot connect to the metastore, which could be the reason for the slow performance and error messages. You can check the eventNodeDiskSpace metric in usage_logs to see if the root file system of the driver's host VM is getting 100%, which can cause the cluster to go down.

Additionally, it is recommended to convert the table to a delta table to improve performance in Unity Catalog. You can also try increasing the number of user connections from the RDS and performing instance-type upgrades.

If the issue persists, you can check the node daemon log of the problematic instance and raise the case to the Dataplane team.

[alesventus]

Hi Kaniz. Thanks for reply. Just want to add some information. Im not sure if this error message is related to legacy Hive metastore or to Unity Catalog. We use UC from the beginning. Our metastore is on external storage - see screenshot.

This error occurs in the clusters event log exactly 6 minutes after the start of the cluster. Every time we start the cluster that error is there.

[karthik_p]

@alesventus what is size of your data and what type of cluster size you are using. also is your workspace region and metastore region same

[Giri-Patcham]

@alesventus I think "Metastore is down"  here is related to the legacy Hive metastore. You can find the stack trace in the driver logs. 
In your case do you need Hivemetastore?

[alesventus]

This issue is solely related to the VNET. Azure engineer must set up connection within VNET correctly.