Connecting to Azure SQL from Azure Databricks with firewalls
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2021 03:35 PM
We are trying to connect to an Azure SQL Server from Azure Databricks using JDBC, but have faced issues because our firewall blocks everything. We decided to whitelist IPs from the SQL Server side and add a public subnet to make the connection work.
Is this a good practice? Is it secure? Is there a better way?
- Labels:
-
Azure
-
Azure databricks
-
Azure sql database
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2021 03:39 PM
Using subnets for Databricks connectivity is the correct thing to do. This way you ensure the resources (clusters) can connect to the SQL Database. We also recommend using NPIP (No Public IPs) so that there won't be any public ip associated with the cluster nodes. This wouldn't matter for the SQL Database but it is more for the peace of mind.
It may be better to have something like an Azure fire wall in front have the Databricks vnet and then allow this firewall on the Azure SQL Database, rather than the subnets which allows you to control egress from Databricks subnets and restricts who gets to talk to Azure SQL from the Databricks perspective.
There are a lot of options here. Feel free to reach out to sales@databricks.com to get connect to an architect.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-30-2022 12:28 PM
Is there a way to achieve this , if both the resources, i.e. databricks cluster and SQL server are in two different regions. Apparently, subnet whitelisting will not work in this case.
