cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Connecting to Azure SQL from Azure Databricks with firewalls

Ryan_Chynoweth
Esteemed Contributor

We are trying to connect to an Azure SQL Server from Azure Databricks using JDBC, but have faced issues because our firewall blocks everything. We decided to whitelist IPs from the SQL Server side and add a public subnet to make the connection work.

Is this a good practice? Is it secure? Is there a better way?

2 REPLIES 2

Ryan_Chynoweth
Esteemed Contributor

Using subnets for Databricks connectivity is the correct thing to do. This way you ensure the resources (clusters) can connect to the SQL Database. We also recommend using NPIP (No Public IPs) so that there won't be any public ip associated with the cluster nodes. This wouldn't matter for the SQL Database but it is more for the peace of mind.

It may be better to have something like an Azure fire wall in front have the Databricks vnet and then allow this firewall on the Azure SQL Database, rather than the subnets which allows you to control egress from Databricks subnets and restricts who gets to talk to Azure SQL from the Databricks perspective.

There are a lot of options here. Feel free to reach out to sales@databricks.com to get connect to an architect.

Is there a way to achieve this , if both the resources, i.e. databricks cluster and SQL server are in two different regions. Apparently, subnet whitelisting will not work in this case.​

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group