[Action required] Azure Databricks is updating the public IP range for its control plane services

[AlexEsibov]

Starting on 30 May 2024, Azure Databricks will begin using new control plane service components. This will affect the egress and ingress IP addresses Azure Databricks’ control plane uses. If you use a firewall or proxy appliance to restrict user access to Azure Databricks control plane, and/or for controlling outbound access to your resources, you will need to update your access rules to include the new IP addresses. Otherwise, user access to the Azure Databricks control plane, including the web app, may be blocked and Azure Databricks’ control plane access to your resources may be blocked.

Please have the admin in your organization responsible for network security for the Azure Databricks platform review the information below. 

What's changing

To support infrastructure improvements, Azure Databricks is deploying new components for our control plane service. These changes will improve multi-zonal availability and routing infrastructure for our web app and control plane. 

Beginning on 30 May 2024, this means that:

1. We will update the ingress  Azure Databricks Control Plane Public IPs and associated Azure Service Tags. These are the IP addresses listed for each region under “Control Plane IPs, including webapp”.
2. We will update the egress Azure Databricks Control Plane Public IPs and associated Azure Service Tags. These are the IP addresses listed for each region under the Service “Control Plane NAT”. 

Action required

1. If you have access rules on your firewall or gateway appliance to restrict access to Azure Databricks Control Plane ingress IPs, you will need to update the rules to include the new IP addresses for your region. See link in resources.
   • Note that if you use Azure Service tags in your Azure Firewall, we will update the existing service tags to include the new IP addresses and no action is required with regards to #1. If you are using Azure Firewall but not leveraging service tags today, we recommend you migrate to using service tags if possible. See link in resources.
2. If  you have access rules in any resource firewall or proxy that restrict access from Azure Databricks Control Plane egress IPs, you will need to update the rules to include the new “Control Plane NAT” IP addresses. See link in resources.

Resources

• The list of all Azure Databricks control plane ingress IP addresses that must be included in your user/service access policies can be found here, under “Control Plane IPs, including webapp” for your region(s)
• The list of all Azure Databricks control plane egress IP addresses that must be included in your resource firewall policies can be found here, under “Control Plane NAT” for your region(s)
• More information on using Azure Service Tags can be found here

If you have any questions or require additional support, please comment below or open a support ticket with Azure Databricks.

Thank you,

Azure Databricks

[ninguemNatural]

Our organization received this alert, but we were unsure about the configurations, would these releases be in the route table attached to the subsnets, or would they be released at the level of NSGs?

[NadithK]

Hi,
At our organization, we are using Vnet injected workspaces with backend privatelink for control plan to compute plane networking (with SCC enabled). We don't have any NSG rules which blocks any traffic at that private endpoint.

A front end private link is enabled for user to workspace web app access and there are not firewall rules which block that either.

In that case, my understanding is that this change will not have no impact on our system. 

Please correct me if I am wrong. Thanks.

[AlexEsibov]

@NadithK correct - if you are using frontend private link, the public IPs for ingress to the Databricks Control Plane won't be relevant. Backend private link here does not map to the egress IPs, because those are for resource access that you have have restricted access to from the control plane (e.g., GitHub); but if you have not put in restrictions like this, no action is needed. 

[AlexEsibov]

@ninguemNatural I'm not sure I understood the question so please correct me if I answer the wrong one - but 

* the new IPs will be added via service tag (see documentation above)

* you would see the new IPs as updated (new) DNS resolutions for your existing workspaces hostnames

[VladimirD1]

Hello,

I didn't see any explicit information so I'll ask.

Currently, the NSGs that are managed by DataBricks instances contains IP addresses of the control plane, NOT service tags.
Do I understand correctly that Databricks will automatically add new IP addresses to the NSGs or switch them to using the tag service?

Thank you!

[AlexEsibov]

@VladimirD1 apologies for the delay - for workspaces that are both:

a) in a locked virtual network in your Azure subscription (i.e., NOT VNet injection workspaces) 

b) NOT using secure cluster connectivity

there is an NSG inbound rule for the compute plane that allows a static list of Databricks control plane IPs that cannot be altered. For now, you can ignore updating these inbound NSG rules specifically. We'll follow up with a separate communication targeting customers with this set-up (a+b), and how to make these changes.