cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

network security perimeter post-setup questions

Rjdudley
Honored Contributor

I've set up the network security perimeter with Terraform using Configure an Azure network security perimeter for Azure resources - Azure Databricks | Microsoft Lea... as my guide.  All of the resources associated to the NSP, and the Status of each is "Succeeded (View issues)"

Rjdudley_0-1780421763241.png

When I view the issue, it says

Provisioning state
Succeeded

Issue type
MissingIdentityConfiguration

Description
Enabling a managed identity (MI) is required to support intra-perimeter communication between resources. Only requests authenticated using MI are permitted for intraโ€‘perimeter access. While some capabilities for certain resources may continue to function without a MI, enabling one is strongly recommended to ensure secure access within the same perimeter or across linked perimeters.

Suggested access rules
None

Suggested resource IDs
None

Suggested fix
Enable managed identity (MI) to ensure this resource can securely access other resources within the same perimeter or across linked perimeters. This is recommended even though certain capabilities may still operate without one.

I didn't see anything in the docs one way or the other for this, is it a problem or can we ignore it?

Following the "Verify ..." step, I get this response (the path was copied from the table's Details)

[RequestId={guid} ErrorClass=INVALID_PARAMETER_VALUE.LOCATION_OVERLAP] Input path url 'abfss://{container}@{storage account}.dfs.core.windows.net/__unitystorage/schemas/{guid}/tables/{guid}' overlaps with managed storage within 'CheckPathAccess' call.

It looks like that is working, but then again, no discussion one way or the other.  Is this expected since all our data access us through UC tables and therefore all locations are UC managed?

1 REPLY 1

Louis_Frolio
Databricks Employee
Databricks Employee

Greetings @Rjdudley , I did some digging and here is what I found:

  1. Good news on both counts. Neither of these means your NSP setup is broken.

    On the MissingIdentityConfiguration issue, you can ignore it for what you're doing. It's a generic Azure NSP advisory, not a Databricks problem, which is why you didn't find it called out in the docs. Azure wants any resource that calls other resources inside or across the perimeter to authenticate with a managed identity, and it nudges you toward that by default. The tell is right there in the issue details: provisioning state is Succeeded, and the suggested access rules, resource IDs, and fix all come back as None. Nothing is actually being blocked, and Azure's own wording says some capabilities keep working without an MI.

    The reason it doesn't apply to you is the direction of traffic. The guide has you create a single inbound rule (service tag AzureDatabricksServerless.<region>) so serverless can reach your storage. Your storage account is on the receiving end. Serverless reaches in to read, your storage isn't reaching out to call other perimeter resources, and outbound resource-to-resource calls are exactly what the MI recommendation is about. It also helps that you're in Transition mode, which evaluates the NSP rules first and then falls back to the resource's existing firewall, so legitimate traffic isn't denied while you validate. So leave it for now. Enable a system-assigned managed identity later, if and when you add perimeter resources that have to authenticate to each other.

    On the LOCATION_OVERLAP error, yes, that's expected, and you've got the reason right: it's because everything goes through UC managed storage. This one is a Unity Catalog guardrail, not an NSP failure. The path you copied sits under __unitystorage/.../tables/..., which is UC managed storage, and UC deliberately refuses direct cloud-path access to managed locations. It rejects the request up front in that CheckPathAccess call, before any network read happens, so the error tells you nothing about NSP either way. The Verify step's delta.abfss://... example is meant for a normal external ADLS path, not a managed-table path pulled from the table's Details tab.

    When everything is UC-managed, validate a different way. Query the table by name on serverless, something like SELECT * FROM catalog.schema.table LIMIT 10. Serverless still reads the underlying files over the network, so a clean result confirms the inbound NSP rule is doing its job. The delta.abfss://.../__unitystorage/... form will always fail no matter how your networking is set up. If you'd rather use the doc's direct-path test, point it at an external location path outside __unitystorage. To confirm at the network layer, use the guide's Monitor NSP activity step: send Diagnostic settings (StorageRead and StorageWrite) to Log Analytics and check whether each request was allowed by the NSP rule or by the resource firewall.

    Short version: the MI warning is safe to ignore for serverless reaching storage, and the LOCATION_OVERLAP error is UC protecting its managed storage, exactly as you'd expect. Validate by querying the table by name and watching the NSP logs, not by hitting a managed path.

    Hope this helps, Louis.