Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-24-2021 06:28 AM
Databricks by default uses hive metastore version 0.13. However, it's possible to switch to a newer client version connecting to the internal Databricks metastore.
Details are available here:
https://docs.databricks.com/data/metastores/external-hive-metastore.html#spark-configuration-options
Set spark.sql.hive.metastore.version to the version of your Hive metastore and spark.sql.hive.metastore.jars as follows:
Hive 0.13: do not set spark.sql.hive.metastore.jars.
Hive 1.2.0 or 1.2.1 (Databricks Runtime 6.6 and below): set spark.sql.hive.metastore.jars to builtin.When it comes to external metastore, based on the use case and need a version can be selected. We recommend checking the compatibility matrix before deciding on the version.