cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Azure Databricks - ADLS Gen 2.0 Access

dalion
New Contributor III

Hi all,

I have a Azure Databricks Setup (non-premium) and an ADLS Gen 2.0 setup. I am trying to access the ADLS Gen 2.0 containers via a simple access key mode for testing.

There is no error, if the ADLS Gen 2.0 is set to "Enable from all networks".

But, when it is set to "Enabled from selected virtual and IP Addresses" and "Allow Azure Services on the trusted services list to access this storage account", its doesn't work.

It errors with 403 Get operation error.

Operation failed: "This request is not authorized to perform this operation.", 403, GET, https://xxxxxxxxx.dfs.core.windows.net/raw?upn=false&resource=filesystem&maxResults=5000&timeout=90&..., AuthorizationFailure, "This request is not authorized to perform this operation. RequestId:342de772-c01f-0036-5a58-7553e4000000 Time:2023-04-22T20:24:04.6283599Z"

5 REPLIES 5

-werners-
Esteemed Contributor III

if you want to put access restrictions on the ADLS, there is some config to do:

https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal

Basically you need to tell what is allowed and what isn't.

dalion
New Contributor III

Hi @Werner Stinckens​  , I have already tried to all the respective IP address. Still its not working.

-werners-
Esteemed Contributor III

the Databricks VM's which are provisioned do not have static ip adresses; so you probably have to keep on adding IPs.

Better to use a subnet mask or use a VNet.

fabio2352
Contributor

dalion
New Contributor III

Hi @Fabio Santos​ , thanks for the response. All these ADLS Gen 2.0 network setting are set to public. But to secure the customer's data, it should contain a list of Whitelisted IP addresses.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!