- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-20-2022 02:06 AM
I have created a premium cluster on Azure site. There were no problem Data Science and Engineering (DSAE) while i was binding Postgre SQL hive metastore. I have done all settings via Global init Scripts from Admin Console.
However when I try to adjust same setting for Databricks SQL, I am getting Error:
Org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientThe setting as written below(Data Access Configuration). I cannot see any other logs from Databricks SQL, I cannot sniff any other causes.
spark.hadoop.fs.azure.account.auth.type.DataLake.dfs.core.windows.net OAuth
spark.hadoop.fs.azure.account.oauth.provider.type.DataLake.dfs.core.windows.net org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider
spark.hadoop.fs.azure.account.oauth2.client.id.DataLake.dfs.core.windows.net {{secrets/Veloxity/VeloxityDateLakeApplicationID}}
spark.hadoop.fs.azure.account.oauth2.client.secret.DataLake.dfs.core.windows.net {{secrets/Veloxity/VeloxityDataLakeSecretID}}
spark.hadoop.fs.azure.account.oauth2.client.endpoint.DataLake.dfs.core.windows.net https://login.microsoftonline.com/{{secrets/Veloxity/VeloxityTenantID}}/oauth2/token
spark.hadoop.javax.jdo.option.ConnectionURL jdbc:postgresql://#########.postgres.database.azure.com:5432/databricks_metastore?ssl=true
spark.hadoop.javax.jdo.option.ConnectionUserName ########@##########.postgres.database.azure.com
spark.hadoop.javax.jdo.option.ConnectionPassword #########
spark.hadoop.javax.jdo.option.ConnectionDriverName org.postgresql.Driver
spark.sql.hive.metastore.jars maven
spark.sql.hive.metastore.version 3.1.0
spark.hadoop.datanucleus.fixedDatastore false
spark.hadoop.hive.metastore.schema.verification falseI have read question written by @[prasadvaze] and answers from @Prabakar Ammeappin however i could not find solution yet. I have change "..metastore.jar maven" or tried to update version nothing has worked yet.
Could you please help me ?
- Labels:
-
Databricks SQL
-
Global init scripts
-
SQL