article [Action required] [Azure] update outbound connectivity for classic workspaces in Product Platform Updates

[Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Wed, 16 Oct 2024 16:38:20 GMT

The following blog post details action required for workspaces that use both secure cluster connectivity and workspace IP access lists. The communication will also be sent out by email to users who Azure Databricks determines might be affected.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

OvZ — Mon, 03 Jun 2024 08:23:17 GMT

Hi, is there a place in the databricks logs where I can see if this switch is enabled (other then in de ui)>> Enforce IP ACL on Dataplane Requests ? and second databricks uses NAT or private link connection ? thx in adv, Oscar

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

Said — Tue, 04 Jun 2024 11:12:24 GMT

I have a doubt on this. If i routed databricks traffic through Azure firewall, whether i need to whitelist the Firewall public IP or is there anything else i have to consider.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Tue, 04 Jun 2024 15:51:37 GMT

@Said good question - you should allow-list the public IPs associated with the outbound traffic from the Databricks compute plane. Depending on the specific proxy/firewall appliance and configuration, some will obfuscate the source IPs, while others can preserve them. If traffic egressing from your Azure firewall uses the public IP of the firewall, I would make sure to allowlist that IP in the workspace IP ACL.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

chang_vnext — Fri, 07 Jun 2024 00:37:53 GMT

Hi, say for example, we have two types of databricks workspace being used. The configurations are

1. Premium tier, with

- IP access list enabled

- Public Network access enabled

- Required NSG Rules: All Rules

- Private endpoint: ui_api service only

2. Standard tier, with

- IP access list disabled

- Public Network access enabled

- Required NSG Rules: All Rules

With these two configurations, do we still need to do anything to ensure the these two workspaces are still working as expected?

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

yankaiZhang-db — Wed, 12 Jun 2024 15:53:51 GMT

@chang_vnext Hi, only the (1) bucket will potentially be affected. Please check the "enableNoPublicIp" parameter of your Azure Databricks workspace. If for your premium tier workspace (1) you have enableNoPublicIp = true, then this workspace is affected and please add the public IP of your NAT gateway to your workspace IP access list, then turn on "Enforce IP access list on Compute Plane Requests" to secure your workspace.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Tue, 18 Jun 2024 22:12:11 GMT

@OvZ sorry I missed your questions:

1) To check if the switch is enabled, you could check logs but using a service 'workspace', action of 'setSetting' and tags.settingTypeName of 'enforce_ip_acl_on_dp'.

2) Your classic compute resources can connect to the Azure Databricks Control Plane over NAT or Private Link - it depends on your deployment.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

elvisleung — Fri, 28 Jun 2024 02:47:14 GMT

Is there any way to check what IP is used from the compute plane to control plane, for example, logs?

In my case, the traffic going out from the compute plane SNet is not going through NAT gateway or firewall. I believe that some sort of Azure public IP will be used. I would like to confirm what are the exact IP(s) that I need to whitelist.

The workspace is with SCC and IP whitelist on and without backend private endpoint.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

Said — Wed, 03 Jul 2024 12:34:57 GMT

I understand that the change will primarily affect workspaces utilizing both secure cluster connectivity (no public IP) and workspace IP access lists. In my Azure Databricks workspace, I'm currently only using secure cluster connectivity, and I do not have workspace IP access lists configured.

My question is: Will this upcoming change regarding workspace IP access controls impact my environment in any way?

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Thu, 04 Jul 2024 18:54:14 GMT

@Said this change will not impact you. You must meet both conditions as you pointed out.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

rugger-bricks — Thu, 04 Jul 2024 20:25:40 GMT

There is a simple way to check your outbound IP from the Databricks compute plane - its a great way to validate a NAT gateway or FW NAT IP .. also can tell you the default SNAT ip if you are NPIP and going direct to the internet.

In a notebook cell run :

%sh curl ipinfo.io | grep ip

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Thu, 04 Jul 2024 21:39:10 GMT

@elvisleung please see the response above from @rugger-bricks and let us know if it resolves your question

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

MarkusL — Tue, 09 Jul 2024 08:42:56 GMT

Hi, we are affected by this change. We are using classic compute with vnet injection. But we does not have a NAT as explained above nor the backend private link.

If I will go for option 1 by adding the compute plane IP to the IP access list, what IP-address should I add? Is it the addresses of my VNET that I have injected?

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Tue, 09 Jul 2024 16:59:28 GMT

hi @MarkusL please deploy a NAT gateway, or configure backend private link. If you go with the former, the instructions above include a step for "Deploy one or more Azure NAT Gateways, if one doesn’t exist already". Let me know if you have any questions.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

Said — Fri, 12 Jul 2024 13:24:46 GMT

Hi,

After enabling the IP access list for my databricks workspace, CI/CD pipeline between the DevOps repo and Databricks is failing. The IPs that are failing are those I added to the access list, but all failing IPs are dynamic. How can I mitigate this issue? TIA

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Fri, 19 Jul 2024 15:52:31 GMT

@Said to be clear - this communication was scoped only to customers who are already using workspace IP ACLs already. That said, it's hard to know exactly why your scenario is failing - if you have a support subscription, you can follow the steps here to submit a support ticket. That would probably be the easiest way to diagnose the issue: https://docs.databricks.com/en/resources/support.html. You can also submit a support case by emailing help@databricks.com.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

Avvar2022 — Mon, 05 Aug 2024 20:04:03 GMT

Our databricks set up is VNET injection and SCC, enabled private link as simplified deployment. as per the implementation i believe we have one front-end and back-end one Browser authentication private endpoint.

i believe we won't be impacted but wanted to get some inputs from community

Enable Azure Private Link as a simplified deployment - Azure Databricks | Microsoft Learn

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

AlexEsibov — Mon, 05 Aug 2024 20:22:25 GMT

@Avvar2022 correct - if you already have backend private link configured, you do not need to take action on that workspace.

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

MarkusL — Thu, 15 Aug 2024 09:37:48 GMT

@AlexEsibov we are going to implement the nat-gateway solution here. I have two questions:

1. We have 3 workspaces in dev and 3 in prod. Should we use a unique public-ip/nat-gateway per each workspace or should/could they share the same? The 3 workspaces are devided in to two vnets, please see below clarification:

vnet1
- workspace 1
- workspace 2
vnet2
- workspace 3

2. Should we NAT the traffic from both the public and private cluster subnets, or is it only the public clusters that communicates with the control plane?

Re: [Action required] [Azure] update outbound connectivity for classic workspaces

Abadoom — Wed, 21 Aug 2024 06:49:59 GMT

Hello,

In my company we have Databricks deployed in Azure with Simplified terraform deployment. We have the two private endpoints for databricks_ui_api and second browser_authentication. As far as I understand, we don't have to make any updates regarding the security update on 26th August. Is that right, because we are little worried? Thank you!