To improve security for Azure Databricks customers, we’ll begin applying workspace IP access controls to compute plane traffic. This change will impact workspaces that use both secure cluster connectivity (no public IP) and workspace IP access lists. We’ll begin enforcing this change for all new workspaces starting July 29 2024 and all existing workspaces starting August 26 2024.
Required action
To ensure there’s no disruption to connectivity to the Azure Databricks control plane, you’ll need to take one of the following actions:
If you are not the admin responsible for network connectivity to Azure Databricks, please forward this email to that person.
Note that while this change only impacts secure cluster connectivity workspaces that use workspace IP access lists, Microsoft has announced that default outbound access for VMs in Azure will be retired on 30 September 2025. Therefore, we recommend proactively taking action.
Help and support
If you have questions, get answers from community experts in Microsoft Q&A. If you have a support plan and you need technical help, open the Azure portal and select the question mark icon at the top of the page.
Option 1) Add your compute plane IP addresses to the workspace IP access list
Note: If your compute plane traffic egresses through a firewall/proxy appliance, ensure that the IPs of the appliance are added to the workspace IP ACL policy. If it does not, read on for Azure NAT gateway deployment.
Note 2: Azure charges for Azure NAT Gateway. See pricing details here.
az network vnet subnet show
--resource-group <resource group>
--vnet <vnet name>
--name <public subnet name>
--query "natGateway.id"
(/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/brn/providers/Microsoft.Network/natGateways/[NAT_gateway_name])
az network nat gateway show --resource-group <resource group> --name <nat-gateway name> --query publicIpAddresses[0].id
(/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/brn/providers/Microsoft.Network/publicIPAddresses/[NAT_gateway_name]))
az network public-ip show
--resource-group <resource group>
--name <public-ip name>
--query "{fqdn: dnsSettings.fqdn, address: ipAddress, type: publicIPAllocationMethod}"
Example : az network public-ip show --resource-group brn --name [NAT_gateway_name]) --query "{fqdn: dnsSettings.fqdn, address: ipAddress, type: publicIPAllocationMethod}"
{
"address": "[IP_address]",
"fqdn": null,
"type": "[e.g., Static]"
}
az network public-ip show
--resource-group <resource group>
--name <public-ip name>
--query "{fqdn: dnsSettings.fqdn, address: ipAddress, type: publicIPAllocationMethod}"
Cell #1
%pip install databricks-sdk --upgrade
dbutils.library.restartPython()
Cell #2
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
w.clusters.list()
If the code sample works, then your IP access list is set up correctly.Optional - Use Azure virtual network service endpoints to access storage. To avoid using NAT for outbound connectivity for accessing storage, you can optionally deploy Azure virtual network service endpoints.
In the Azure portal, go to the Databricks workspace object, click on “see more” and take note of the public subnet name.
Click on the Virtual network. open the public (host) subnet for you workspace and find the config entry “service endpoints”
In the services drop down choose between “Micrososft.Storage” (for in region service endpoint networking) or “Microsoft.Storage.Global” (for cross region service endpoint networking) Adding service endpoint networking for Databricks public subnet. Note: this approach has the following important limitations:
Option 2) Configure back-end private link for all workspaces, if not already done
Note: Azure charges for Azure Private Link. See details here.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.