NAT Gateway with Azure Databricks

[ph1l0s0ph3r]

Hi all, what are the steps to enable external communication through a NAT Gateway? Our Databricks instance was created through VNET injection and did not have a NAT gateway created by default. We now want to pass external traffic through NAT gateway rather than the Databricks IP. Thanks

[szymon_dybczak · Accepted solution]

Hi @ph1l0s0ph3r ,

NAT Gateway is created by default only if you use SCC with default VNet that Azure Databricks creates. Look at below entry in documentation:

- Secure cluster connectivity - Azure Databricks | Microsoft Learn

In you case, when you have deployed Databricks instance into your own VNet you need to configure NAT Gateway by yourself. 
Basically, you have following option to configure egress with NPIP in VNet Injected workspace:

1. Egress Load Balancer
2. Azure NAT Gateway
3. Azure Firewall

You're interested in option 2 - Azure NAT Gateway. So to set it up you need  to create:

- NAT Gateway in Azure

-  configure the gateway on both of the workspace’s subnets to ensure that all outbound traffic to the Azure backbone and public network transits through it.

[ph1l0s0ph3r]

Thank you, @szymon_dybczak ! I will try this and get back on this thread.

Is a route table necessary for allowing outbound traffic in the subnets at all? Or will just adding the NAT gateway to both the subnets be sufficient?

[szymon_dybczak]

Adding the NAT Gateway to both subnet should be sufficient. If you encounter any problem let us know 😉

[ph1l0s0ph3r]

I made the changes and it works! I was able to communicate to the NAT Gateway IP in our external on-prem application.

I noticed though that Databricks instance is not showing the NAT Gateway within Parameters. Is this an issue?