cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Trying to mount Azure Data Lake Storage Gen 2 to Azure Databricks

ChrisS
New Contributor III

I have validated many many many times all my credentials and I am still getting the following error (at the very end). ChatGPT said to basically recheck everything and I did. The one thing I didn't do was grant the permissions which I have since done and I am still having problems. Here is what I have done so far:

Ensure that the storage_account_name variable is correctly assigned with the name of your Azure Storage account. Double-check that the variable is defined and has the correct value.

Verify that the mount_point variable is defined and specifies a valid mount point path in Azure Databricks. The mount point should start with /mnt/ followed by a unique name or path.

Confirm that the configs dictionary variable is defined and populated with the necessary configuration settings. Ensure that all the required keys and values are present in the dictionary.

Check that the Azure AD application credentials (client ID, client secret, directory ID) used in the configs dictionary are valid and have the necessary permissions to access the Azure Data Lake Storage Gen2 account.

Any other ideas on what I can do? Thank you

Error:

java.lang.NullPointerException

---------------------------------------------------------------------------

ExecutionError Traceback (most recent call last)

File <command-3548438897654738>:19

10 configs = {

11 "fs.azure.account.auth.type": "OAuth",

12 "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",

(...)

15 "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/59bfdcb9-d634-447e-88ab-e561deb22047/oauth2/token"

16 }

18 # Mount the storage account

---> 19 dbutils.fs.mount(

20 source=f"abfss://{storage_account_name}.dfs.core.windows.net/",

21 mount_point=mount_point,

22 extra_configs=configs

23 )

File /databricks/python_shell/dbruntime/dbutils.py:362, in DBUtils.FSHandler.prettify_exception_message.<locals>.f_with_exception_handling(*args, **kwargs)

360 exc.__context__ = None

361 exc.__cause__ = None

--> 362 raise exc

2 REPLIES 2

Anonymous
Not applicable

Hi @Chris Sarrico​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.

User16752245772
Contributor

Hi @Chris Sarrico​ 

Could you please specify the container name before the storage account name like this :

so the source looks like this. :

source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/"

https://learn.microsoft.com/en-us/azure/databricks/dbfs/mounts#--mount-adls-gen2-or-blob-storage-wit...

I hope this will work

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.