parameterize azure storage account name in spark cluster config databricks

baatchus
New Contributor III

wondering if this is to parameterize the azure storage account name part in the spark cluster config in Databricks?

I have a working example where the values are referencing secret scopes:

spark.hadoop.fs.azure.account.oauth2.client.id.<azurestorageaccountname>.dfs.core.windows.net {{secrets/keyvault_secret_scope/sp-id}}
spark.hadoop.fs.azure.account.oauth.provider.type.<azurestorageaccountname>.dfs.core.windows.net org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider
spark.hadoop.fs.azure.account.oauth2.client.endpoint.<azurestorageaccountname>.dfs.core.windows.net https://login.microsoftonline.com/05646f53-5e81-4a6a-85f5-a1ee2189bea4/oauth2/token
spark.hadoop.fs.azure.account.auth.type.<azurestorageaccountname>.dfs.core.windows.net OAuth
spark.hadoop.fs.azure.account.oauth2.client.secret.<azurestorageaccountname>.dfs.core.windows.net {{secrets/keyvault_secret_scope/sp-secret}}

But this does not work if I try it on the keys:

spark.hadoop.fs.azure.account.oauth2.client.id.{{secrets/keyvault_secret_scope/azurestorageaccountname}}.dfs.core.windows.net {{secrets/keyvault_secret_scope/sp-id}}
 
spark.hadoop.fs.azure.account.oauth.provider.type.{{secrets/keyvault_secret_scope/azurestorageaccountname}}.dfs.core.windows.net org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider
 
spark.hadoop.fs.azure.account.oauth2.client.endpoint.{{secrets/keyvault_secret_scope/azurestorageaccountname}}.dfs.core.windows.net https://login.microsoftonline.com/05646f53-5e81-4a6a-85f5-a1ee2189bea4/oauth2/token
 
spark.hadoop.fs.azure.account.auth.type.{{secrets/keyvault_secret_scope/azurestorageaccountname}}.dfs.core.windows.net OAuth
 
spark.hadoop.fs.azure.account.oauth2.client.secret.{{secrets/keyvault_secret_scope/azurestorageaccountname}}.dfs.core.windows.net {{secrets/keyvault_secret_scope/sp-secret}}

If it is impossible how can you replace the cluster config between environments?

I need the azure storage account name to be different in DEV vs PROD.