topic Re: Unable to read data from ADLS using databricks serverless sql pool in Data Engineering

Unable to read data from ADLS using databricks serverless sql pool

NarenderKumar — Thu, 14 Mar 2024 12:08:50 GMT

I have a data bricks workspace and an Azure data lake storage account.

Both are present in the same Vnet.

Unity catalog is enabled in the worksapce.

I have created some tables in unity catalog.

I am able to query the data from the tables when I use the any of the below 2 computes:

1. All purpose cluster

2. SQL Warehouse of type "PRO"

When I try to read the same table data using a SQL warehouse of the type "serverless", I get the below error:

This Azure storage request is not authorized. The storage account's 'Firewalls and virtual networks' settings may be blocking access to storage services. Please verify your Azure storage credentials or firewall exception settings.

I know that the access, firewalls and networking are not a problem because it is working with all purpose cluster and PRO SQL warehouse. So I not know what is the problem here.

Please let me how to fix this.

Re: Unable to read data from ADLS using databricks serverless sql pool

Hubert-Dudek — Thu, 14 Mar 2024 12:28:16 GMT

but serverless is in Databricks Subscription so you need to whitelist traffic from there.

Please check that list. Storage firewall needs to allow connections https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewall

Re: Unable to read data from ADLS using databricks serverless sql pool

BhawaniD — Mon, 21 Oct 2024 15:13:31 GMT

I am having the same issue. How did you resolve it?

Re: Unable to read data from ADLS using databricks serverless sql pool

saiV06 — Thu, 12 Dec 2024 15:01:25 GMT

I'm having the same issue and tried to follow the document shared above, but quite not sure what I'm missing, as I can't make it work. Can someone please help me here? TIA.