Assistance Required: Issues Creating External Table in Legacy Hive Metastore

jv_v — Thu, 03 Oct 2024 10:39:42 GMT

I am currently trying to create an external table with an external location in the legacy Hive metastore. As part of this process, I have also created the necessary secret scope using below steps, but I am still encountering issues when attempting to create the table.

steps used to create a scope:

Step 1: databricks secrets create-scope --scope myScope

Step 2: databricks secrets put-secret myScope storage_account_key --string-value "My Storage Account Key"

Step 3: Create table with below script

CREATE External TABLE if not exists hive_metastore.test_ext.employees (

employee_id INT,

first_name STRING,

last_name STRING,

email STRING,

phone_number STRING,

hire_date DATE,

job_id STRING,

salary DECIMAL (10, 2),

commission DECIMAL (5, 2),

manager_id INT,

department INT

)

USING csv

LOCATION 'abfss://demorkco@demorkstr.dfs.core.windows.net/employees.csv';

Error: Failure to initialize configuration for storage account demorkstr.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.key

Could you please assist me in troubleshooting this matter? Any guidance on the correct steps or configurations would be greatly appreciated.

Re: Assistance Required: Issues Creating External Table in Legacy Hive Metastore

gchandra — Thu, 03 Oct 2024 11:09:03 GMT

Did you mount the Location using the secrets?

https://learn.microsoft.com/en-us/azure/databricks/dbfs/mounts#mount-adls-gen2-or-blob-storage-with-abfs

topic Assistance Required: Issues Creating External Table in Legacy Hive Metastore in Data Engineering

Assistance Required: Issues Creating External Table in Legacy Hive Metastore

Re: Assistance Required: Issues Creating External Table in Legacy Hive Metastore