topic Spark configuration to access Azure Blob or ADLS in Get Started Discussions

Spark configuration to access Azure Blob or ADLS

yugz — Thu, 26 Jun 2025 20:36:21 GMT

I am new to Databricks, I am using the free edition of Databricks. I have tried
[spark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net", "SAS")
spark.conf.set("fs.azure.sas.token.provider.type.<storage-account>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider")
spark.conf.set("fs.azure.sas.fixed.token.<storage-account>.dfs.core.windows.net", dbutils.secrets.get(scope="<scope>", key="<sas-token-key>"))] and [spark.conf.set(
"fs.azure.account.key.<storage-account>.dfs.core.windows.net",
dbutils.secrets.get(scope="<scope>", key="<storage-account-access-key>"))] but I keep getting the same error

[CONFIG_NOT_AVAILABLE] Configuration fs.azure.account.key.ygustorage.dfs.core.windows.net is not available. SQLSTATE: 42K0I

Re: Spark configuration to access Azure Blob or ADLS

Renu_ — Fri, 27 Jun 2025 11:07:29 GMT

Hi @yugz, this looks similar to an issue discussed in another thread (https://community.databricks.com/t5/data-engineering/access-adls-with-serverless-config-not-available-error/td-p/111417), could you try the solution mentioned there and see if it helps?

Re: Spark configuration to access Azure Blob or ADLS

szymon_dybczak — Fri, 27 Jun 2025 14:03:20 GMT

Hi @yugz ,

This is depracted approach to configure storage in Databricks. If you're using Databricks Free edition your new workspace includes serverless compute and default storage, so you can immediately start exploring and building on Databricks. The point is - you don't have to configure storage, because it's already been configure for you.

Re: Spark configuration to access Azure Blob or ADLS

yugz — Fri, 27 Jun 2025 15:01:10 GMT

Hello @szymon_dybczak ,

Thanks for reaching out to my concern. I am just been introduced to Databricks. How can I access the data I have in my bronze folder on ADLS?

Re: Spark configuration to access Azure Blob or ADLS

nayan_wylde — Fri, 27 Jun 2025 15:15:53 GMT

This is an old way of accessing data lake. With the free edition and serverless it is not supported. Try creating an external credential and external location to the data lake.

https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-external-locations

Re: Spark configuration to access Azure Blob or ADLS

szymon_dybczak — Fri, 27 Jun 2025 15:36:24 GMT

Hi,

I don't know if you can create external location in databricks free edition, but this would be my approach ( assuming you want to bring in your own storage account):

https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-external-locations

Here's a really good video guide that explains those concepts and shows how to configure it:

https://youtu.be/kRfNXFh9T3U

Re: Spark configuration to access Azure Blob or ADLS

yugz — Fri, 27 Jun 2025 18:07:15 GMT

The only options here are for AWS or Cloudflare to create credentials

Re: Spark configuration to access Azure Blob or ADLS

yugz — Fri, 27 Jun 2025 18:07:46 GMT

The only options here are for AWS or Cloudflare to create credentials

Re: Spark configuration to access Azure Blob or ADLS

nayan_wylde — Fri, 27 Jun 2025 19:55:29 GMT

Check out this

https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/storage-credentials

Re: Spark configuration to access Azure Blob or ADLS

yugz — Fri, 27 Jun 2025 20:00:11 GMT

All I see is for AWS and Cloudflare and nothing for Azure cloud. I am using Azure cloud.