Data bricks + Network security perimeter (storage account) Error

[MyProfile]

Serverless Compute + NCC configration in account console + Private Endpoint (PE) to storage account is working as expected.

When Network security perimeter (NSP) is added and storage account (SA), in NSP we have 2 access mode to choose (transition  and enforced )

1). Serverless + NCC + PE and Storage account (SA) Public access disabled - Working

2). Serverless + NCC + PE + NSP (transition) and Storage account (SA) Public access disabled -working

ERROR :

3). Serverless + NCC + PE + NSP (either transition  or enforced) and Storage account (SA) network access selected as  Secured by perimeter - Not Working - Error 

simple select query  :

select * from databricks_training.training.employees returns error-
[UNAUTHORIZED_ACCESS] Unauthorized access: PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None SQLSTATE: 42501
Note  : In NSP inbound rule, there is option to select service tag, i have selected both global and regional serverless service tag.

My understanding is that the data plane communication should occur through the private endpoint. However, I am unsure whether any control plane communication is also being initiated and getting blocked, which could be causing the issue.

At this point, I do not have complete clarity on the exact root cause. Additionally, I anticipate that a similar issue may also occur with Classic Compute clusters in the same setup.

See attachment for configurations

Assistance Needed:

Is there any mitigation or supported configuration available when NSP is configured on a Storage Account and the storage account access level is set to “Secured by perimeter,” while accessing it from Databricks clusters?

Could you please help clarify how this scenario should be configured to avoid the access error?

Additionally:

Is the issue occurring because control plane communication is being blocked?
Since the “Allow Azure Databricks Control Plane” option via service tags is not available in this configuration, is there any alternative configuration or recommended approach that would work in this setup?

[Louis_Frolio]

Hey @MyProfile , I looked into your issue and here is what I found: 

Short version: your instinct is right.

The data read over the private endpoint is fine. What's failing is a storage account level call, the Get User Delegation Key operation that Unity Catalog runs through your Access Connector's managed identity to mint the short lived SAS it hands to compute. That call rides on the storage account's own firewall exception for trusted services and resource instances. The moment you flip the account to "Secured by perimeter," the NSP becomes the sole gatekeeper and that firewall exception is no longer honored, so the key request gets denied (SQLSTATE 42501). Private endpoint traffic is never subject to NSP, which is exactly why the data path looks healthy but the query dies before it reads anything.

Why your three cases behave the way they do

The decisive setting isn't transition versus enforced. It's whether the storage account's public network access stays on your own firewall config, or gets set to "Secured by perimeter." That's the reason Case 3 fails in both modes.

1. Case 1 (private endpoint plus firewall locked down, no NSP): the Get User Delegation Key call is allowed by the storage firewall's resource instance and trusted Azure services exception for your Access Connector. Works.
2. Case 2 (private endpoint plus NSP in transition, account keeps its own firewall): transition mode falls back to the existing firewall rules and trusted services settings when no NSP rule matches, so that same exception still allows the key request. Works.
3. Case 3 (private endpoint plus NSP plus account set to "Secured by perimeter"): you've handed all public network gatekeeping to the NSP, so there's no firewall ruleset left to fall back to. Microsoft states that under perimeter governance even Azure's trusted service exceptions aren't honored. The serverless service tags you added cover serverless compute egress, not this credential minting path, which is why adding them changed nothing. Fails.

Answering your questions directly

Is control plane communication being blocked? Effectively yes. The managed identity credential vending call is. Let me be honest about my confidence here. The docs clearly establish three things: the failing operation is Get User Delegation Key, perimeter governance bypasses the trusted services and firewall exceptions, and the AzureDatabricksServerless tag doesn't cover this path. Those three facts fully explain your symptom. What the public docs do not literally state is "the Databricks control plane is blocked," and there is no "Allow Databricks control plane" service tag. So treat the control plane framing as a strongly supported inference, and confirm it with NSP diagnostic logs (more on that below).

Is there a supported configuration with "Secured by perimeter"? Not a documented one today. Databricks' own NSP guidance is to stay in transition mode (your working Case 2) and it explicitly calls enforced or perimeter governed access not recommended for most customers, advising you to remain in transition mode to keep compatibility with your existing network setup. So Case 2 is the supported pattern, not a degraded fallback.

Any alternative, since there's no control plane service tag? The only identity based option NSP gives you is a subscription based inbound rule, which Microsoft documents as allowing inbound access authenticated using any managed identity from the subscription. If "Secured by perimeter" is a hard mandate, the thing to test is a subscription based inbound rule for the subscription that holds your Access Connector. I'd put that forward as something to try, but it's unverified for this exact path. Confirm it with diagnostic logs and a support case before you rely on it.

Checklist before you conclude it's purely NSP

1. Rule out RBAC as a quick sanity check. The same error text appears when the Access Connector's identity lacks the Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action permission, meaning the Storage Blob Delegator role granted at storage account (or resource group, or subscription) scope, not just container scope. Since your Cases 1 and 2 work, this is almost certainly already correct, but it's worth confirming so you don't chase the wrong thing.
2. Confirm both private endpoints exist. You need a dfs and a blob private endpoint, plus approved connections, for the storage account.
3. Check which storage account this is. Your query hits databricks_training.training.employees. If that catalog lives in your workspace storage account (the managed dbstorage account), note that enabling firewall support makes Databricks auto provision its own NSP allowing AzureDatabricksServerless. If you want to manage your own perimeter, you can detach the Databricks provisioned one and attach yours, though the switchover causes a brief outage. A custom "Secured by perimeter" config layered on top of the Databricks provisioned one can conflict. If it's a customer managed UC storage account, this auto NSP doesn't apply, but the credential vending diagnosis above still holds.

Recommended path

Best supported option today: private endpoints (dfs plus blob) plus storage firewall locked down (public access disabled) plus NSP associated in transition mode, without switching the account to "Secured by perimeter." That's your working Case 2: strong isolation, no public data path, and it's what Databricks recommends.

If "Secured by perimeter" is mandatory: test the subscription based inbound rule for the Access Connector's subscription, validate with NSP diagnostic logs (StorageRead and StorageWrite) to confirm the Get User Delegation Key request is being allowed, and open a joint Microsoft and Databricks support case, since this exact combination isn't documented as supported yet.

On Classic compute, your prediction is correct. UC credential vending is identical regardless of compute type, so "Secured by perimeter" would break Classic the same way. On top of that, Classic isn't covered by AzureDatabricksServerless, so its data path needs its own VNet and network rules. A serverless only fix won't carry over.

Takeaway: this is NSP governance, not your private endpoint or NCC setup. The failing step is the user delegation key call, and "Secured by perimeter" is what removes the firewall exception that call depends on. Stay in transition mode unless you're forced off it, and if you are, validate everything against the diagnostic logs before you trust it.

Cheers, Louis.

Sources used to help with my research:

1. Configure an Azure network security perimeter for Azure resources, Azure Databricks: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serv... (transition versus enforced, AzureDatabricksServerless tag scope, the "remain in transition mode" recommendation).
2. Network Security Perimeter for Azure Storage, Microsoft Learn: https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security-perimeter (NSP overrides firewall, trusted service exceptions not honored under perimeter, private endpoint traffic exempt, subscription based rule and managed identity access).
3. Enable firewall support for your workspace storage account, Azure Databricks: https://learn.microsoft.com/en-us/azure/databricks/security/network/storage/firewall-support (dfs and blob private endpoints required, Databricks auto onboards the workspace storage account to its own NSP, detach to bring your own).
4. Community thread, "Request for user delegation key is not authorized": https://community.databricks.com/t5/administration-architecture/permission-denied-request-for-user-d... (the same error from the RBAC and Storage Blob Delegator angle, useful for ruling that out).