Alright, we've implemented a workaround for this, and so far it's been working very well:

• First, we created a reusable notebook to wait until Hive has been initialized (see code below).
• We then execute this notebook using the %run command at the top of any notebook which is encountering the Hive issue.

Here is the code:

import time
 
retries = 0
max_retries = 10
while True:
  try:
    # Use this table to check if Hive is ready, since it's very small & all in 1 file
    table("database.small_table")
    break
  except Exception as e:
    if retries == max_retries:
      raise e
      
    retries += 1
    print(f"Hive is not initialized yet. Retrying in 60 seconds. (Retry #{retries})")
    time.sleep(60)
    
print("Hive is initialized!")

And here is what the output looks like: