Connecting to Blob storage using abfss not working with serverless compute

[Mathias]

I tried to follow the instructions found here: Connect to Azure Data Lake Storage Gen2 and Blob Storage - Azure Databricks | Microsoft Learn

E.g. this code:

spark.conf.set(
"fs.azure.account.key.<storage-account>.dfs.core.windows.net",
dbutils.secrets.get(scope="<scope>", key="<storage-account-access-key>"))

Doesn't work and gives error:

Configuration fs.azure.account.key.mystorageaccount.dfs.core.windows.net is not available.

Is there any plans to support this with serverless compute?

[szymon_dybczak]

Hi @Mathias ,

This is legacy way of configuring storage? Is there any reason why you're not using Unity Catalog storage credentials?

[Mathias]

Can you point me to some documentation on how to do that?

[szymon_dybczak]

Hi @Mathias ,

Sure, here it is: 

https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/