Hello All,
I get the org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient while trying to create a database
scripts used
%sql
USE hive_metastore;
CREATE DATABASE XYZ;
%sql
CREATE DATABASE XYZ;
%sql
CREATE DATABASE hive_metastore.XYZ;
the warehouse seems to be in started state
Cluster details
Alright, we've implemented a workaround for this, and so far it's been working very well:
- First, we created a reusable notebook to wait until Hive has been initialized (see code below).
- We then execute this notebook using the %run command at the top of any notebook which is encountering the Hive issue.
Here is the code:
import time
retries = 0
max_retries = 10
while True:
try:
# Use this table to check if Hive is ready, since it's very small & all in 1 file
table("database.small_table")
break
except Exception as e:
if retries == max_retries:
raise e
retries += 1
print(f"Hive is not initialized yet. Retrying in 60 seconds. (Retry #{retries})")
time.sleep(60)
print("Hive is initialized!")And here is what the output looks like:
