[Action required] [GCP] update outbound connectivity for classic workspaces

[AlexEsibov]

IMPORTANT NOTE: We have indefinitely delayed the automatic enforcement described below for workspaces that had enabled workspace IP access lists prior to July 29, 2024. We still recommend manually enforcing IP access lists on compute plane requests in these workspaces by taking the steps outlined below. 

Note: New IP access controls enabled on workspaces after July 29, 2024 are still enforced on data plane traffic, per the original communication below.

---------------------------------

Communication

To enhance security, we are making a change to workspaces that use the workspace IP access lists feature. For these workspaces, we will begin to apply workspace IP access controls to compute plane traffic. See Action Required and Timeline below for details.

Action Required

This change will impact all new workspaces on July 29 2024, and existing workspaces on August 26 2024. 

To ensure your compute plane can continue to talk to the Databricks control plane - take action to add the Public NAT IP addresses to your workspace IP access list. Step-by-step instructions are available below.

Timeline

The required actions must be taken by the following dates:

• Starting on July 29 2024, all new workspaces that use workspace IP access lists will begin enforcing workspace IP access lists on compute plane traffic
• By August 26 2024, all existing workspaces that use workspace IP access lists will begin enforcing workspace IP access lists on compute plane traffic

Step-by-Step Instructions

Note: If your compute plane traffic egresses through a firewall/proxy appliance, ensure that the IPs of the appliance are added to the workspace IP ACL policy. If it does not, read on for Cloud NAT deployment.

1. Make sure your Cloud NAT is allocated a static public IP
   1. Please follow the instructions here: 
      https://cloud.google.com/nat/docs/set-up-manage-network-address-translation#specify_ip_addresses_for... 
2. Retrieve IPs for cloud NAT
1. Via GCP console 
   1. Go to https://console.cloud.google.com/net-services/nat/list?project=<your_project_id>
   2. Select the NAT gateway that is used by your workspace. If the workspace uses DB managed VPC, the NAT gateway will follow the naming pattern "databricks-<workspace_id>-nat"
   3. Find "Cloud NAT IP addresses"
   4. Copy the IP address 
   5. If there are multiple NAT gateways deployed (e.g., for multiple zones), collect all IP addresses for the NAT gateway
2. Via CLI (cloud shell)
   1. Get the public IP resource ID

      gcloud compute routers describe <NAME_OF_CLOUD_ROUTE> --project=<GCP_PROJECT_ID> --region=<REGION> --format='get(nats[].natIps)'
      

   2. Get the public IP address from the resource

      gcloud compute addresses describe <PUBLIC_IP_ADDRESS_NAME> --project=<GCP_RPOJECT_ID> --region=<REGION> --format='get(address)'

1. Add the NAT Gateway IP addresses to the workspace IP access list
1. Follow the steps outlined here to add the IP addresses for the NAT gateways collected above to your workspace IP ACL policy:
   https://docs.gcp.databricks.com/en/security/network/front-end/ip-access-list-workspace.html
2. Test that your deployment was successful
1. Log in to your workspace
2. Navigate to "Preview" > "View All" 
3. Find "Enforce IP access list on Compute Plane Requests". On toggle on, IP ACL will be enforced on your NAT IP
4. Wait for up to 10 minutes for the config to be applied to the workspace.
5. Create and run a python notebook with a new cluster of any type except serverless.

   Cell #1 

   %pip install databricks-sdk --upgrade
   dbutils.library.restartPython()
   

   Cell #2

   from databricks.sdk import WorkspaceClient
   
   w = WorkspaceClient()
   w.clusters.list()
   

   If the code sample works, then your IP access list is set up correctly.
6. In case of failures, toggle off "Enforce IP access list on Compute Plane Requests" to further troubleshoot. Wait for up to 10 minutes for the config to be applied to the workspace.