AWS Firewall Websocket Settings

[crvkumar]

Hello,

Our architecture has customer-managed VPC which sits behind the AWS Firewall. I followed the databricks documentation and I created a stateful domain rule group for .cloud.databricks.com but I am not clear about creating outbound Websocket opening to *.cloud.databricks.com. 

With just domain rule group, clusters are not running(data plane can not communicate with control plane).

It would be helpful if anyone can guide on how to enable outbound databricks websocket communication in AWS Firewall.

[Kaniz_Fatma]

Hi @crvkumar, To enable outbound WebSocket communication for Databricks in your AWS Firewall, you have a couple of options.

Let’s explore them:

1. *Option 1: Allow Traffic to .cloud.databricks.com

   • This option is more permissive but easier to configure. You’ll need to update your firewall rules to allow HTTPS and WebSocket traffic to *.cloud.databricks.com.
   • By doing this, you cover all Databricks workspaces in your account without the need to update rules for each individual workspace.
   • However, keep in mind that this approach allows traffic to all Databricks subdomains under cloud.dat...¹.

2. Option 2: Allow Traffic to Your Databricks Workspaces and Account Console Only

   • If you prefer a more specific configuration, you can set up firewall rules for each workspace in your account.
   • Here’s how:
     • Identify Your Workspace Domains:
       • Your Databricks workspace uses two domain names:
         • The first domain is the one you use to log in (e.g., yourcompany.cloud.databricks.com if you have a vanity domain or dbc-<random-string>.cloud.databricks.com if not).
         • To find the second domain, log in to the first domain. After logging in, check your browser address bar. You should see a URL like https://<first-domain>/?o=<workspace-id>, where <workspace-id> is a string of digits.
         • If you don’t see a ?o= followed by a string of digits in the URL, contact your Databricks account team to get your workspace ID. The second domain has the format dbc-dp-<workspace-id>.cloud.databricks.com. For example, if the workspace ID is 123456, your second domain is dbc-dp-123456.cloud.databricks.com.
         • If you need to access the account console from that network, also allow traffic to accounts.cloud.databricks.com.
     • Update Your Firewall Rules:
       • Update your firewall rules to allow HTTPS and WebSocket traffic to the two domains identified in the previous step.
   • This approach provides more granular control but requires additional configuration for each workspac...¹.

Remember to adjust your firewall rules based on your specific requirements and security policies. By allowing the necessary traffic, you’ll ensure that your Databricks clusters can communicate effectively between the data plane and control plane. 🚀

[crvkumar]

Hello Kaniz,

Thanks for the reply. But this is exactly what is in the Databricks documentation.

Can you elaborate on how to set up firewall rules to allow Websocket traffic in AWS Firewall? Clearly creating only stateful rule group allowing .cloud.databricks.com is not sufficient.

Regards,

Venki.