Error "Invalid configuration value detected for fs.azure.account.key" when listing files stored in an Azure Storage account using "dbutils.fs.ls"

[Mado]

I get the following error when getting a list of files stored in an Azure Storage account using "dbutils.fs.ls" command in Databrciks.

Failure to initialize configuration for storage account AAAAA.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.keyInvalid configuration value detected for fs.azure.account.key

 

Databricks Workspace:

• Unity catalog is enabled.
• There is a catalog named "Exploration"
• Service principal has "ALL PRIVILEGES" grant on this catalog

 

Azure storage account: ADLS Gen2

Service principal has Read/Write/Execute access to the root folder. Also, it has the same access to the sub-folder "test".

 

What should I do to be able to get a list of files?

Notes:

• I am able to create sub-folders in the Exploration storage location.
• Also, I can upload files into the storage location.
•   I can get the list of files in the "raw" container using dbutils:

 

Configurations to access the Azure data lake storage are done by another company. I think they have used "External Locations" in Databricks.

[Anonymous]

Hi @Mohammad Saber​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.

[Mado]

I solved the issue by creating an External Location in Databricks workspace.