Serverless Compute + NCC configration in account console + Private Endpoint (PE) to storage account is working as expected.
When Network security perimeter (NSP) is added and storage account (SA), in NSP we have 2 access mode to choose (transition and enforced )
1). Serverless + NCC + PE and Storage account (SA) Public access disabled - Working
2). Serverless + NCC + PE + NSP (transition) and Storage account (SA) Public access disabled -working
ERROR :
3). Serverless + NCC + PE + NSP (either transition or enforced) and Storage account (SA) network access selected as Secured by perimeter - Not Working - Error
simple select query :
select * from databricks_training.training.employees returns error-
[UNAUTHORIZED_ACCESS] Unauthorized access: PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None SQLSTATE: 42501
Note : In NSP inbound rule, there is option to select service tag, i have selected both global and regional serverless service tag.
My understanding is that the data plane communication should occur through the private endpoint. However, I am unsure whether any control plane communication is also being initiated and getting blocked, which could be causing the issue.
At this point, I do not have complete clarity on the exact root cause. Additionally, I anticipate that a similar issue may also occur with Classic Compute clusters in the same setup.
See attachment for configurations
Assistance Needed:
Is there any mitigation or supported configuration available when NSP is configured on a Storage Account and the storage account access level is set to โSecured by perimeter,โ while accessing it from Databricks clusters?
Could you please help clarify how this scenario should be configured to avoid the access error?
Additionally:
Is the issue occurring because control plane communication is being blocked?
Since the โAllow Azure Databricks Control Planeโ option via service tags is not available in this configuration, is there any alternative configuration or recommended approach that would work in this setup?
