External location + Failure to initialize configuration for storage account

[RYBK]

Hello,
I created a storage credential and an external location. Test is ok, I'm able to browse it from the portal. 
I have a notebook to create a table :
%sql
CREATE OR REPLACE TABLE myschema.mytable
(
  data1 string,
 data2 string
)
USING DELTA LOCATION "abfss://mycontainer@myaccount.blob.core.windows.net/";
 
When I'm executing the notebook with a SQL warehouses cluster, it works fine ; with a standard cluster it failed with this error "Failure to initialize configuration for storage account myaccount.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.key"
 
I thought it was no longer needed to set fs.azure.account.key. with an external location, am I wrong ? Am I missing something ?
 

[RYBK · Accepted solution]

Hello @Retired_mod 

Thank you for your response. We found the issue. We use a ML Cluster with "no isolation shared" as access mode. This cluster is not compatible with unity. We set fs.azure.* variables in the cluster conf. like below to fix our issue:

fs.azure.account.oauth2.client.id.mydatalake.dfs.core.windows.net: {{secrets/mysecret/myclientid}}. 

Is there a better/proper way to do it ?

Regards

[RYBK]

I believe this is what we are doing, as I mentioned in my example, right?

Thank you