topic Re: Access azure storage account from databricks notebook using pyspark or SQL in Data Engineering

Access azure storage account from databricks notebook using pyspark or SQL

Ambi — Mon, 04 Apr 2022 16:34:14 GMT

I have a storage account - Azure BLOB Storage

There I had container. Inside the container we had a CSV file. Couldn't read the file using the access Key and Storage account name.

Any idea how to read file using PySpark/SQL? Thanks in advance

Re: Access azure storage account from databricks notebook using pyspark or SQL

Hubert-Dudek — Mon, 04 Apr 2022 16:46:32 GMT

@Ambiga D , How to mount Azure Data Lake Storage - https://community.databricks.com/s/feed/0D53f00001eQGOHCA4

How to mount blob storage (remember to allow network access to storage from your databricks workspace):

configs = {"fs.azure.account.key.YOUR_STORAGE.blob.core.windows.net" : "YOUR_KEY"}
 
dbutils.fs.mount(
  source = "wasbs://YOUR_CONTAINER@YOUR_STORAGE.blob.core.windows.net",
  mount_point = "/mnt/YOUR_FOLDER",
  extra_configs = configs)

Re: Access azure storage account from databricks notebook using pyspark or SQL

Ambi — Tue, 05 Apr 2022 12:40:31 GMT

@Hubert Dudek , Thank you for the the solution. Couldn't create the private endpoint in the network access. We will work on this and update you.

Re: Access azure storage account from databricks notebook using pyspark or SQL

Hubert-Dudek — Mon, 18 Apr 2022 09:40:08 GMT

Hi, have you managed to mount?

Re: Access azure storage account from databricks notebook using pyspark or SQL

Atanu — Fri, 13 May 2022 05:47:23 GMT

@Ambiga D you need to mount the storage https://docs.databricks.com/data/data-sources/azure/azure-storage.html#mount-azure-blob-storage-containers-to-dbfs you can follow this,thanks.

Re: Access azure storage account from databricks notebook using pyspark or SQL

jose_gonzalez — Tue, 07 Jun 2022 18:15:36 GMT

Hi @Ambiga D,

Just a friendly follow-up. Did any of our previous replies help you to resolved this issue? Please let us know if you still need help