NAT gateway with public IP for SCC disabled Databricks cluster

[vsd]

Hi Team,

We need to have single public IP for all outbound traffic flowing through our Databricks cluster. The Secure Cluster Connectivity (SCC) is disabled for our cluster and currently we get dynamic public IPs assigned to the VMs under managed resource group.

Can we explicitly configure NAT gateway (with a public IP) for a Databricks cluster having SCC disabled? Or does it have any dependency on SCC?

Thank you!

[szymon_dybczak]

Hi @vsd ,

I think so. If your workspace is VNET injected, then you can deploy your own NAT Gateway and redirect all outbound traffic from subnets through UDR to that gateway.

[vsd]

Thank you @szymon_dybczak! I was not able to find any reference documentation which mentions this very clearly. Do you have any reference for configuring such setup?

[szymon_dybczak · Accepted solution]

Hi, 

I don't think you will find any reference at MS docs. This is custom setup. But really similar approach is done in below knowledge base entry. In this article they choose to use firewall as stable egrees IP, but it doesn't matter. The same kind of steps you need to perform to make it work with your own NAT Gateway. Basically, you need to route all outbound traffic from databricks subnets to NAT Gateway 

https://kb.databricks.com/cloud/azure-vnet-single-ip

[vsd]

Thank you @szymon_dybczak ! That's helpful!

[szymon_dybczak]

No problem @vsd, I'm glad that it help you 🙂