Connecting to Azure SQL from Azure Databricks with firewalls

[Ryan_Chynoweth]

We are trying to connect to an Azure SQL Server from Azure Databricks using JDBC, but have faced issues because our firewall blocks everything. We decided to whitelist IPs from the SQL Server side and add a public subnet to make the connection work.

Is this a good practice? Is it secure? Is there a better way?

[Ryan_Chynoweth]

Using subnets for Databricks connectivity is the correct thing to do. This way you ensure the resources (clusters) can connect to the SQL Database. We also recommend using NPIP (No Public IPs) so that there won't be any public ip associated with the cluster nodes. This wouldn't matter for the SQL Database but it is more for the peace of mind.

It may be better to have something like an Azure fire wall in front have the Databricks vnet and then allow this firewall on the Azure SQL Database, rather than the subnets which allows you to control egress from Databricks subnets and restricts who gets to talk to Azure SQL from the Databricks perspective.

There are a lot of options here. Feel free to reach out to sales@databricks.com to get connect to an architect.

[Dbks_Community]

Is there a way to achieve this , if both the resources, i.e. databricks cluster and SQL server are in two different regions. Apparently, subnet whitelisting will not work in this case.​