Due to the need for Azure storage private endpoints, we switched our databricks deployment to use an injected vnet. Now, when our deployment pipeline tries to re-create the workspace (e.g. az databricks workspace delete), it seems to leave the MS created resource group along with the Network policies associated with it.
Given that we are now deploying using the az deployment group create command, what is the proper way to wipe the databricks resources so we can reset the environment and install from scratch?
Error: ERROR: {"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"BadRequest","message":"{\r\n \"error\": {\r\n \"code\": \"ConflictWithNetworkIntentPolicy\",\r\n \"message\": \"Found conflicts with NetworkIntentPolicy. Details: Network Security Group cannot have resources which conflict with its subnets' network intent policies.\\r\\nNetwork Security Group: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkSecurityGroups/databricks-nsg conflicts with Network Intent Policy: adb-usgovvirginia-5be1a1d89639b052ed24de46\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-ssh, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-5be1a1d89639b052ed24de46/securityRules/databricks-control-plane-to-worker-ssh, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 22\\r\\n ----\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-proxy, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-5be1a1d89639b052ed24de46/securityRules/databricks-control-plane-to-worker-proxy, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 5557\\r\\n ----\\r\\n---- ----\\r\\nNetwork Security Group: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkSecurityGroups/databricks-nsg conflicts with Network Intent Policy: adb-usgovvirginia-7f6098ea7d9303d6d4585e01\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-ssh, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-7f6098ea7d9303d6d4585e01/securityRules/databricks-control-plane-to-worker-ssh, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 22\\r\\n ----\\r\\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-control-plane-to-worker-proxy, Id: /subscriptions/bb1f0029-311f-4f1a-a63b-a5c3c2575782/resourceGroups/orcus-usgovvirginia/providers/Microsoft.Network/networkIntentPolicies/adb-usgovvirginia-7f6098ea7d9303d6d4585e01/securityRules/databricks-control-plane-to-worker-proxy, Access: Allow, Direction: Inbound, Protocol: tcp, SourceAddressPrefix: AzureDatabricks, SourcePortRange: *, DestinationAddressPrefix: VirtualNetwork, DestinationPortRange: 5557\\r\\n ----\\r\\n---- ----\",\r\n \"details\": []\r\n }\r\n}"}]}}
