Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

Mado — Wed, 21 Jun 2023 11:39:48 GMT

I get the following error when getting a list of files stored in an Azure Storage account using "dbutils.fs.ls" command in Databrciks.

Failure to initialize configuration for storage account AAAAA.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.keyInvalid configuration value detected for fs.azure.account.key

Databricks Workspace:

Unity catalog is enabled.
There is a catalog named "Exploration"
Service principal has "ALL PRIVILEGES" grant on this catalog

Azure storage account: ADLS Gen2

Service principal has Read/Write/Execute access to the root folder. Also, it has the same access to the sub-folder "test".

What should I do to be able to get a list of files?

Notes:

I am able to create sub-folders in the Exploration storage location.
Also, I can upload files into the storage location.
I can get the list of files in the "raw" container using dbutils:

Configurations to access the Azure data lake storage are done by another company. I think they have used "External Locations" in Databricks.

Re: Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

Anonymous — Thu, 22 Jun 2023 04:54:41 GMT

Hi @Mohammad Saber

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.

Re: Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

Mado — Thu, 22 Jun 2023 07:51:43 GMT

I solved the issue by creating an External Location in Databricks workspace.

topic Re: Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls" in Machine Learning

Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

Re: Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

Re: Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"