Azure Databricks: Required change in resource firewalls allowlisting Databricks serverless compute

AlexEsibov
Databricks Employee
Databricks Employee
‎01-26-2026 11:56 AM

Background

To improve serverless scalability and expand service endpoint support to additional resources, Azure Databricks will make a change to the identifiers used to firewall storage access from serverless compute. 

If you allow-list Azure Databricks serverless subnet IDs in any Azure storage firewalls, action is required. Note that if you disallow public access to your storage accounts and use Azure Private Link to connect from Azure Databricks, no action is required.

Following a comprehensive review, we have identified additional affected subscriptions. To ensure a unified transition across all impacted customers, the migration deadline has been updated to June 9, 2026.

Required Action

By June 9, 2026, any existing Azure storage account allowlisting Databricks serverless subnet IDs must: 

  1. Be onboarded to a network security perimeter in transition mode 
    1. If you are unable to onboard to a network security perimeter, you must reach out to file a support ticket to discuss alternatives by 30 March 2026.  
  2. In your network security perimeter firewall(s), allowlist AzureDatabricksServerless. Regional scoping is recommended (e.g., AzureDatabricksServerless.EastUS2).
Note: Using the AzureDatabricksServerless service tag and allows Azure Databricks serverless compute to communicate with your Azure resources over the Azure backbone. The tag maps to Azure Databricks public IPs that represent service endpoints and NAT IPs.

Important callout: Remain in transition mode indefinitely to avoid impact to your storage access, if any of the Network Security Perimeter limitations apply to you. 

Additional details, including the benefits of this change, and step-by-step guidance, can be found here.

Help and support 

If you have questions, get answers from community experts in Microsoft Q&A. If you have a support plan and need technical help, please create a support request.

Benefits of NSP

For Azure Databricks serverless outbound traffic, today’s subnet ID feature enables customers to connect over service endpoints to in-region or paired-region Azure storage accounts. This access has no data processing charge, and stays on the Azure backbone. The migration to Network Security Perimeter will enable Databricks to add additional resource support for service endpoints in the future, saving significant data processing charges for customers, and improving security posture.

Instructions

Please refer to public documentation for step-by-step instructions: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serv...

Automation Tools for Migration

If you would like helping to automate migration of your storage accounts, we have created a public repo that takes subscriptions as inputs, and enables retrieving storage accounts configured with Databricks serverless subnet IDs, and creating or updating Network Security Perimeters (NSPs) with the needed policy. Please refer to: 

https://github.com/brucenelson6655/nsp-migrate 

Helpful Links:

Quickstart - Create a network security perimeter - Azure PowerShell - Azure Private Link | Microsoft... 

Quickstart - Create a network security perimeter - Azure CLI - Azure Private Link | Microsoft Learn

Comments
ittzzmalind
New Contributor III
‎05-21-2026 10:28 AM
‎05-21-2026 10:28 AM

Hi Team,

im facing difficulty in configuring NSP 

Serverless Compute + NCC configration in account console + Private Endpoint (PE) to storage account is working as expected.

When Network security perimeter (NSP) is added and storage account (SA), in NSP we have 2 access mode to choose (transition  and enforced )

1). Serverless + NCC + PE and Storage account (SA) Public access disabled - Working

2). Serverless + NCC + PE + NSP (either transition  or enforced) and Storage account (SA) Public access disabled -working

ERROR :

3). Serverless + NCC + PE + NSP (either transition  or enforced) and Storage account (SA) network access selected as  Secured by perimeter - Not Working - Error 

simple select query  :

select * from databricks_training.training.employees returns error-
[UNAUTHORIZED_ACCESS] Unauthorized access: PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None SQLSTATE: 42501
Note  : In NSP inbound rule, there is option to select service tag, i have selected both global and regional serverless service tag.

My understanding is that the data plane communication should occur through the private endpoint. However, I am unsure whether any control plane communication is also being initiated and getting blocked, which could be causing the issue.

At this point, I do not have complete clarity on the exact root cause. Additionally, I anticipate that a similar issue may also occur with Classic Compute clusters in the same setup.

See attachment for configurations

 

Assistance Needed:

Is there any mitigation or supported configuration available when NSP is configured on a Storage Account and the storage account access level is set to “Secured by perimeter,” while accessing it from Databricks clusters?

Could you please help clarify how this scenario should be configured to avoid the access error?

Additionally:

Is the issue occurring because control plane communication is being blocked?
Since the “Allow Azure Databricks Control Plane” option via service tags is not available in this configuration, is there any alternative configuration or recommended approach that would work in this setup?

MiaPenfoldLopez
Databricks Employee
Databricks Employee
2 weeks ago
2 weeks ago

Hey @ittzzmalind 

Based on the behavior you described, it seems like the issue is caused by setting the Azure Storage account’s Public network access to “Secured by perimeter.” This setting is currently not a supported configuration for Databricks serverless Unity Catalog external locations, even when an NCC private endpoint is configured.

The error below is consistent with this limitation:

PERMISSION_DENIED: Request for user delegation key is not authorized

Unity Catalog requires a user delegation key as part of the credential-vending flow. That request is evaluated separately from the serverless compute data path. Therefore, a healthy NCC private endpoint does not guarantee that the user-delegation-key request will be permitted when the storage account is set to “Secured by perimeter.” Adding the AzureDatabricksServerless service tag does not provide a workaround for this setting.

Recommended configuration

For Azure Storage accessed by serverless compute, use one of these supported patterns:

  • NSP/service-endpoint pattern: keep Public network access set to “Enabled from selected networks,” keep the NSP association in Transition mode, and add the regional AzureDatabricksServerless.<region> service tag to the NSP inbound rule. Use the global tag only when access is required from multiple regions.

  • Private-only pattern: keep Public network access disabled and use NCC-managed private endpoints for the required Storage subresources. Confirm that the private endpoint requests are approved and that the NCC private endpoint rules are in ESTABLISHED state. For ADLS Gen2, validate both dfs and blob endpoints where applicable.

If the NCC private endpoint is retained, it is still important not to change the storage account to “Secured by perimeter.” The current Azure Databricks guidance is to keep the resource on “Enabled from selected networks” after associating it with an NSP. NSP Transition mode evaluates NSP rules first and falls back to the existing resource firewall rules when no NSP rule matches.

Accordingly, the immediate mitigation is:

  • Change the storage account setting from “Secured by perimeter” back to “Enabled from selected networks.”

  • Keep the NSP in Transition mode.

  • Keep the regional AzureDatabricksServerless.<region> rule, if using the NSP/service-endpoint pattern.

  • Confirm the Unity Catalog external location covers the path and that the caller has the required privileges.

  • Enable NSP diagnostic logs and review denied requests while retesting the external location and query.

Classic compute considerations

Classic compute should be validated separately because it runs in the customer-managed VNet rather than the serverless compute plane. If the storage account accepts only private connectivity, classic clusters must use an appropriate VNet private endpoint or service-endpoint design, including DNS, endpoint approval, routing, and the required Azure RBAC permissions for the configured identity. Control-plane/SCC connectivity for classic compute is a separate workspace-network requirement and should be configured using the applicable Azure Databricks service tag, FQDNs, UDRs, or Private Link architecture.

References:

Best regards,

Mia Penfold Lopez