Databricks

TheoDeSo · 07-11-2023

Hello,After implementing the use of Secret Scope to store Secrets in an azure key vault, i faced a problem.When writting an output to the blob i get the following error:shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Unable to access con...

TheoDeSo · 07-12-2023

Hi all thank you for the suggestions. Doing This spark.conf.set("fs.azure.account.key.{storage_account}.dfs.core.windows.net", "{myStorageAccountKey}")For the hadoop configuration does not work.And the suggestion of @Tharun-Kumar would suggest to har...

TheoDeSo · 07-12-2023

Hello unfortunatly this is not the desired solution as this involves hardcoding the secret in the configuration of the cluster. I posted the question on stack overflow https://stackoverflow.com/questions/76655569/databricks-not-allowing-to-write-outp...

TheoDeSo · 07-11-2023

Hello @Vadim1 and @User16764241763. I'm wondering if you find a way to avoid adding the hardcoded key in the advanced options spark config section in the cluster configuration. Is there a similar command to spark.conf.set("spark.hadoop.fs.azure.accou...

Databricks

User Stats

User Activity

Error on Azure-Databricks write output to blob storage account

Re: Error on Azure-Databricks write output to blob storage account

Re: Error on Azure-Databricks write output to blob storage account

Re: Error on Azure-Databricks write RDD to storage account with wsabs://