[sql warehouse] Invalid configuration value detected for fs.azure.account.key with 'force' = 'true'

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

Hello,

I am getting the following error when trying to copy data to databricks from an ADLS with SQL and using a SAS token

Failure to initialize configuration for storage account <storage account>: Invalid configuration value detected for fs.azure.account.key

I am using an SQL Warehouse to copy the data from the ADLS. For this I am using a COPY_INTO.

The following works fine :

COPY INTO `table` FROM ( SELECT _c0 as `c0` FROM 'abfss://<storageaccount>' WITH ( CREDENTIALS (AZURE_SAS_TOKEN = '***') ) ) FILEFORMAT = CSV    FORMAT_OPTIONS (  'delimiter'='\t', 'quote'='"', 'escape'='"', 'multiLine'='true' )  COPY_OPTIONS ('mergeSchema' = 'true')

But this fails :

COPY INTO `table` FROM ( SELECT _c0 as `c0` FROM 'abfss://<storageaccount>' WITH ( CREDENTIALS (AZURE_SAS_TOKEN = '***') ) ) FILEFORMAT = CSV FORMAT_OPTIONS ( 'delimiter'='\t', 'quote'='"', 'escape'='"', 'multiLine'='true' ) COPY_OPTIONS ('mergeSchema' = 'true', 'force' = 'true')

The only difference is in the copy options : 'force' = 'true'

The same SQL instruction works fine on a cluster.

The only place it is failing is on an SQL warehouse when 'force' is true.

Is someone facing the same issue ?

Thanks