topic Trying to create external table in Hive Metastore in Data Engineering

Trying to create external table in Hive Metastore

mayank_gupta — Thu, 04 Jul 2024 13:17:28 GMT

Receiving this error: KeyProviderException: Failure to initialize configuration for storage account adlspersonal.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.key

I used hive meta store to save my table

%python spark.read.table("catalog.default.table").write.option("path", "abfss://container@storage_acc.dfs.core.windows.net/dir1").saveAsTable("annual_enterprise_survey")

But, on running:

%python dbutils.fs.ls("abfss://container@storage_acc.dfs.core.windows.net/dir1")

Things are working perfectly. Why so?

Apart from this when trying to save tables in UC things are again working fine.

Re: Trying to create external table in Hive Metastore

amr — Thu, 04 Jul 2024 17:45:00 GMT

not sure about that error, but try to do it using SQL, see if it will work
create table hive_metastore.annual_enterprise_survey as select * from catalog.defaul.table

Re: Trying to create external table in Hive Metastore

mayank_gupta — Thu, 04 Jul 2024 18:47:32 GMT

I tried that too but received the same error. I am trying to understand the issue here. It says Invalid configuration value detected for fs.azure.account.key. But the same is working when trying to list files.