cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Creating Azure Databricks Workspace Without NAT Gateway

ziad
New Contributor III

Hello,

Recently, when I create a new Databricks workspace on Azure, it automatically create a NAT Gateway which incurs additional cost !

nat 1.jpg

When creating the workspace, I don't choose secure cluster connectivity, so I'm expecting not to have a NAT Gateway created.

nat 2.jpg

Why NAT gateway is created for my Databricks workspaces knowing that it was not the case a couple weeks ago ? and how to delete it, or deploy workspaces without it ?

If this is not possible, how to avoid this extra cost related to NAT Gateway ?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Really weird, could you check how's your arm template looks like in rg deployments? To make sure that in your case there is no scc deployment?
I've created workspace with the same configuration and same region as your and my input for arm template looks like below. Inside managed rg I don't have NAT Gateway.

Slash_0-1721595947401.png

Slash_3-1721596145799.png

 

I've also created another workspace, but this time with SCC enabled. And now I've got NAT Gateway, just as is stated in documentation.

Slash_1-1721596027271.png

Slash_2-1721596106163.png

 

 

View solution in original post

4 REPLIES 4

szymon_dybczak
Contributor III

Hi @ziad ,

Do you create workspace with secure cluster connectivity? According to the documentation

If you use secure cluster connectivity with the default VNet that Azure Databricks creates, Azure Databricks automatically creates a NAT gateway for outbound traffic from your workspaceโ€™s subnets to the Azure backbone and public network. The NAT gateway is created within the managed resource group managed by Azure Databricks. You cannot modify this resource group or any resources provisioned within it.

The automatically-created NAT gateway incurs additional cost.

If you would like to avoid automatic creation of NAT gateway in secure cluster connectivity, you should perform VNET injection. Read below documentation entry for additional details:

Secure cluster connectivity - Azure Databricks | Microsoft Learn

Secure cluster connectivity - Azure Databricks | Microsoft Learn

EDIT: Sorry, didn't notice that you've mentioned you don't use secure cluster connectivity. So that's bit weird. Maybe at your organization there is some kind of azure policy that disables IP (so it enables implicitly SCC) ?

I've also created new workspace with the same configuration as yours and I don't have NAT gateway in DB manged RG


ziad
New Contributor III

Thank you @szymon_dybczak for your reply.

Not only in my organization subscription, but also I've created a new Azure Free Tier account, and it automatically creates NAT Gateway for Databricks workspaces.

Really weird, could you check how's your arm template looks like in rg deployments? To make sure that in your case there is no scc deployment?
I've created workspace with the same configuration and same region as your and my input for arm template looks like below. Inside managed rg I don't have NAT Gateway.

Slash_0-1721595947401.png

Slash_3-1721596145799.png

 

I've also created another workspace, but this time with SCC enabled. And now I've got NAT Gateway, just as is stated in documentation.

Slash_1-1721596027271.png

Slash_2-1721596106163.png

 

 

ziad
New Contributor III

Indeed weird! It seems that the issue has been resolved but I really don't understand what was going on. It seems it's an interface issue in Azure portal. For now, I'll go a head and accept your solution as it helps to better debug the issue. I will update the discussion in case I encountered the issue again. Thank you @szymon_dybczak for your help

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group