I've been trying to use ​the HiveMetastoreClient class in Scala to extract some metadata from Databricks internal Metastore, without success. I'm currently using the 7.3 LTS runtime.

​

The error seems to be related to some kind of inconsistency between Client and Hive Metastore Schema versions, but I can't immediately tell why. By running the following Scala snippet:

%scala 
 
import org.apache.hadoop.hive.metastore.HiveMetaStoreClient
import org.apache.hadoop.hive.conf.HiveConf
 
var hiveConf = new HiveConf
var client = new HiveMetaStoreClient(hiveConf)
client.getTable("my_db", "my_table")

I get the following MetaException

Caused by: javax.jdo.JDOException: Exception thrown when executing query : SELECT DISTINCT 'org.apache.hadoop.hive.metastore.model.MTable' AS NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE A0.TBL_NAME = ? AND B0.`NAME` = ?
NestedThrowables:
java.sql.SQLSyntaxErrorException: (conn=18979393) Unknown column 'A0.IS_REWRITE_ENABLED' in 'field list'

Does anyone know what could be the issue here?

Runtime: 7.3 LTS

Conf:

spark.sql.hive.metastore.version: 0.13.0