Delta sharing open issue with access data on storage

[cszczotka]

Hi, I have configured delta sharing for external consumer in Azure Databricks. Azure Databricks and storage account are in VNET, no public access. The storage account has also disabled account key access and  shared key authorization.

I'm running delta sharing client on local spark cluster and I'm getting such error:

io.delta.sharing.client.util.UnexpectedHttpStatus: HTTP request failed with status: HTTP/1.1 403 This request is not authorized to perform this operation. {"error":{"code":"AuthorizationFailure","message":"This request is not authorized to perform this operation.\nRequestId:68b35ea9-a01f-0042-4d35-9873d0000000\nTime:2025-03-18T18:44:33.2175103Z"}}, while accessing URI of shared table file

Similar issue like here: https://kb.databricks.com/security/cross-cloud-delta-sharing-query-results-in-403-response.

To solve this issue I have to add my delta sharing recipient IP address to azure storage firewall where I have data.

Is it required to open access to the storage account on firewall? Why is not used the Databrick storage account connector but "direct" access to storage account ? How delta sharing authorize to storage account if is disabled account key access and  shared key authorization ?

[mark_ott]

Delta Sharing in Azure Databricks allows sharing datasets across clouds and with external consumers, but when used in a tightly controlled network environment (private endpoints, no public access, restricted storage account authentication), it behaves in ways that can surprise users. Here is a detailed explanation of the scenario you described, addressing your questions directly:

Delta Sharing Data Access Pattern

When a recipient accesses a shared Delta table via Delta Sharing, the data flow is as follows:

• The Delta Sharing server (running on Databricks) generates pre-signed URLs (SAS tokens) pointing to data files in Azure Storage.

• The recipient's Delta Sharing client (running outside Databricks) downloads the data directly from your Azure Storage account using those SAS URLs, not through Databricks APIs or connectors.

Why You See HTTP 403 Authorization Issues

• If your storage account firewall allows only VNET or private endpoint access, but your external consumer (Delta Sharing client) is outside that VNET, the client cannot access the storage account, even with a valid SAS token. This causes the 403 error.

• Disabling "account key" and "shared key" affects how storage is accessed, but SAS-based access (via tokens) is supported as long as the firewall/network rules allow the access from the client's IP address.

Do You Need to Open Storage Firewall for the External Client?

• Yes. To allow the external client to download the shared files, you must permit the client’s public IP in the storage account firewall or set up some network path (e.g., a jump box, VPN, or service endpoint) that grants the client access.

Why Not Use the Databricks Storage Connector?

• Delta Sharing is designed for "cloud-to-cloud" and "beyond workspaces" data sharing, meaning the external client does not run inside your Databricks workspace and does not have access to your secure environment or internal connectors.

• Instead, your client relies on the short-lived access credentials (SAS URLs) to read from your cloud storage directly—this is by design for Delta Sharing (open sharing using "serverless" architecture).

How Does Delta Sharing Authorize to Storage Account?

• Delta Sharing issues the client a SAS token (shared access signature) that allows temporary, scoped access to just the files needed for the query.

• If your storage account firewall blocks the client's IP, or if access by SAS token is not permitted (via firewall or policy), the client cannot read the data, regardless of the SAS token.

Summary Table

Access Factor	Required for Delta Sharing Client?
Public Network/Firewall IP Whitelisting	Yes, client IP must be allowed
Account/Shared Key Authorization	No, not needed; SAS token is used
Databricks Workspace Access	No, external clients are outside workspace
SAS Token/URL	Yes, this is how temporary access is granted

References

• The error and explanation match the scenario in the Databricks Knowledge Base article referenced, which explicitly states that external clients need storage firewall permissions or public access to download shared data.

• Microsoft Azure documentation clarifies that SAS-based access to storage can be blocked by firewall rules even if the SAS is valid.

In conclusion:
You must open up storage firewall access for the external recipient's IP (or provide some network access path) because Delta Sharing uses direct, authorized (via SAS) access to your storage account. This is true even if you have disabled account/shared key access, as those methods are not used by Delta Sharing—but your firewall/network controls still apply and can block external consumers.