I am following Microsoft documentation to connect from Databricks workspace to Azure blob storage. but it is not working. Any help is greatly appreciated. Below is the code
spark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net", "SAS")
spark.conf.set("fs.azure.sas.token.provider.type.<storage-account>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider")
spark.conf.set("fs.azure.sas.fixed.token.<storage-account>.dfs.core.windows.net", dbutils.secrets.get(scope="<scope>", key="<sas-token-key>"))
It gives below error:
Configuration fs.azure.account.auth.type.tmpbricksteststorage1.dfs.core.windows.net is not available. JVM stacktrace: org.apache.spark.sql.AnalysisException at com.databricks.sql.connect.SparkConnectConfig$.assertConfigAllowed(SparkConnectConfig.scala:195) at org.apache.spark.sql.connect.service.SparkConnectConfigHandler$RuntimeConfigWrapper.set(SparkConnectConfigHandler.scala:89) at org.apache.spark.sql.connect.service.SparkConnectConfigHandler.$anonfun$handleSet$1(SparkConnectConfigHandler.scala:223) at org.apache.spark.sql.connect.service.SparkConnectConfigHandler.$anonfun$handleSet$1$adapted(SparkConnectConfigHandler.scala:221) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at
