Connecting to Blob storage using abfss not working with serverless compute
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-04-2024 03:39 AM
I tried to follow the instructions found here: Connect to Azure Data Lake Storage Gen2 and Blob Storage - Azure Databricks | Microsoft Learn
E.g. this code:
spark.conf.set(
"fs.azure.account.key.<storage-account>.dfs.core.windows.net",
dbutils.secrets.get(scope="<scope>", key="<storage-account-access-key>"))
Doesn't work and gives error:
Configuration fs.azure.account.key.mystorageaccount.dfs.core.windows.net is not available.
Is there any plans to support this with serverless compute?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-04-2024 04:57 AM
Hi @Mathias ,
This is legacy way of configuring storage? Is there any reason why you're not using Unity Catalog storage credentials?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-04-2024 05:10 AM
Can you point me to some documentation on how to do that?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-04-2024 08:23 AM
Hi @Mathias ,
Sure, here it is:
https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/
