IMPORTANT NOTE: we have delayed opening additional ports for new workspaces until October 11, 2024. For existing workspaces, Network Security Groups (NSGs) have been updated. If you received a communication that updates for your existing workspaces failed, please read on.
If updates to your existing workspaces failed, you must manually modify your NSGs to allow outbound traffic to the Azure Databricks control plane over 10 new ports: 3306, 8443-8451.
If your NSG is in a locked Azure resource group or subscription, please remove the lock before performing the update. Also, ensure that you are not denying any resource changes to your NSG with an Azure Policy before performing the update.
Please refer to the NSG rules here as a reference: Deploy Azure Databricks in your Azure virtual network (VNet injection)
Note that if you have multiple workspaces using a single NSG, we may have detected a false positive workspace update failure. If the outbound rule to destination "AzureDatabricks (service tag)" contains ports 8443-8451, the update was successfully applied.
----------------------
Original Communication below
Background: To improve scalability and enable new features, the Azure Databricks control plane will communicate with the classic compute plane over 10 new ports.
We’ll be updating your existing network security groups (NSGs) used for the public and private subnets of your VNet injection workspaces with an additional rule to allow outbound traffic to the Azure Databricks control plane over 10 new ports: 3306 and 8443–8451. Existing security rules already allow outbound traffic over port 443 for VNet injection workspaces and, for back-end private link workspaces, port 6666.
Action:
We'll open the new ports on your behalf, and most customers won’t need to take any action. However, if you have existing rules that disallow these ports, you may need to update these rules before this date.
- If you have existing VNet injection workspaces, you’ll need to ensure that no NSGs on your public and private subnets block the new ports.
- If you use back-end private link, ensure that no NSGs on your private endpoints block these ports.
- Ensure no firewall rules block these ports.
- Note that the NSG rules that Azure Databricks auto-provisions have been replaced with equivalent rules with custom names, duplicate rules may be created. This won't affect functionality.
If there is a rule blocking these ports at the time Databricks pushes this update, the update will fail. However, in this event, workspaces will continue to function properly for existing features. You will know that the update failed if the NSG does not have the new ports explicitly included after date of the change.
Timeline:
- For existing workspaces, we will open the new ports on June 3, 2024
- For new workspaces, we will open the new ports on July 1, 2024. Ensure any workspace creation automation is also updated by then.
In both cases, please allow up to 2 weeks for the change to rollout to all regions.
If you are not the admin responsible for network connectivity to Azure Databricks, please forward this email to that person. If you have questions, get answers from community experts in Microsoft Q&A. If you have a support plan and need technical help, please create a support request.
