cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

DBX injected V-Net and Deployment

Dave_B_
New Contributor III

Due to the need for Azure storage private endpoints, we switched our databricks deployment to use an injected vnet. Now, when our deployment pipeline tries to re-create the workspace (e.g. az databricks workspace delete), it seems to leave the MS created resource group along with the Network policies associated with it.

Given that we are now deploying using the az deployment group create command, what is the proper way to wipe the databricks resources so we can reset the environment and install from scratch?

Error: ERROR: {"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"BadRequest","message":"{\r\n \"error\": {\r\n \"code\": \"ConflictWithNetworkIntentPolicy\",\r\n \"message\": \"Found conflicts with NetworkIntentPolicy. Details: Network Security Group cannot have resources which conflict with its subnets' network intent policies.\\r\\nNetwork Security Group: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkSecurityGroups/databricks-nsg conflicts with Network Intent Policy: adb-usgovvirginia-5be1a1d89639b052ed24de46\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-ssh, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-5be1a1d89639b052ed24de46/securityRules/databricks-control-plane-to-worker-ssh, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 22\\r\\n ----\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-proxy, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-5be1a1d89639b052ed24de46/securityRules/databricks-control-plane-to-worker-proxy, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 5557\\r\\n ----\\r\\n---- ----\\r\\nNetwork Security Group: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkSecurityGroups/databricks-nsg conflicts with Network Intent Policy: adb-usgovvirginia-7f6098ea7d9303d6d4585e01\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-ssh, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-7f6098ea7d9303d6d4585e01/securityRules/databricks-control-plane-to-worker-ssh, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 22\\r\\n ----\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-proxy, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-7f6098ea7d9303d6d4585e01/securityRules/databricks-control-plane-to-worker-proxy, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 5557\\r\\n ----\\r\\n---- ----\",\r\n \"details\": []\r\n }\r\n}"}]}}

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, this looks like few of the resources were deleted manually and few left out and hence the error Network Intent Policy: adb-usgovvirginia-5be1a1d89639b052ed24de46 is throwing up.

To delete the Azure Databricks resources, you can refer to https://learn.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/account#del..., which deletes the service along with all the resources inside. And if a workspace needs to be deleted then deleting the workspace from Databricks account console works and deletes the resources in the backend. Please let us know if this helps.

Anonymous
Not applicable

Hi @David Benedict​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!