Failed to Access Azure Storage Container from Databricks in ADF Pipeline

p_a_l — Fri, 14 Nov 2025 09:57:24 GMT

Hi Databricks community,

I'm new to Databricks and I'm trying to connect Databricks to Azure MySQL using Azure Data Factory.

When I run the pipeline, I get the following error:

ErrorCode=AzureDatabricksCommandError, Hit an error when running the command in Azure Databricks. Error details: SparkConnectGrpcException: (shaded.databricks.org.apache.hadoop.fs.azure.AzureException) shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container adfstagedcommandtempdata in account stdeltapipelinesnowflake.blob.core.windows.net not found, and we can't create it using anonymous credentials, and no credentials found for them in the configuration. JVM stacktrace: shaded.databricks.org.apache.hadoop.fs.azure.AzureException Caused by: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container adfstagedcommandtempdata in account stdeltapipelinesnowflake.blob.core.windows.net not found, and we can't create it using anonymous credentials, and no credentials found for them in the configuration. Caused by: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container adfstagedcommandtempdata in account stdeltapipelinesnowflake.blob.core.windows.net not found, and we can't create it using anonymous credentials, and no credentials found for them in the configuration.

I'm using stdeltapipelinesnowflake as my Azure Storage Account.

After following the suggestion from the Databricks Assistant and setting the Spark config for the storage account, I now get this:

[CONFIG_NOT_AVAILABLE] Configuration spark.hadoop.fs.azure.account.key.stdeltapipelinesnowflake.dfs.core.windows.net is not available. SQLSTATE: 42K0I

Has anyone encountered this before and can help me out?

Thanks!

Re: Failed to Access Azure Storage Container from Databricks in ADF Pipeline

Coffee77 — Fri, 14 Nov 2025 10:59:22 GMT

Could you provide more details about high level flow or design you are planning? Not understanding completely but you can take a look here how to connect ADF with Databricks via PATs -> https://www.sqlshack.com/integrating-azure-databricks-with-azure-data-factory/

Re: Failed to Access Azure Storage Container from Databricks in ADF Pipeline

palearnings — Mon, 17 Nov 2025 16:10:58 GMT

Hi Coffee77,

Thanks for getting back to me.
I’ve managed to resolve the issue. It was related to access to the storage account in Azure - I needed to specify it in the config.

Regards,

topic Re: Failed to Access Azure Storage Container from Databricks in ADF Pipeline in Warehousing & Analytics

Failed to Access Azure Storage Container from Databricks in ADF Pipeline

Re: Failed to Access Azure Storage Container from Databricks in ADF Pipeline

Re: Failed to Access Azure Storage Container from Databricks in ADF Pipeline