Azure Databricks Serverless Compute Unable to Connect to Azure SQL MI Using Failover Group FQDN via

[LokeshChikuru]

We are using Azure Databricks Serverless Compute to connect to Azure SQL Managed Instance (SQL MI) through Network Connectivity Configuration (NCC) with Private Endpoint Rules. Connectivity works successfully when using the primary SQL MI FQDN. However, connectivity fails when the SQL Managed Instance is configured behind a Failover Group and the connection is attempted using the Failover Group listener FQDN. As a result, Databricks Serverless Compute is currently unable to establish a connection to Azure SQL MI via the Failover Group FQDN, while direct connections to the primary SQL MI endpoint continue to work as expected.

[balajij8]

Hi Lokesh,

Its likely tied to how Azure Databricks Network Connectivity Configurations handle DNS resolution and Private Endpoint routing relative to Azure SQL Managed Instance Failover Groups.

When you connect using the primary SQL MI FQDN, compute routes directly through the NCC Private Endpoint created for that specific resource ID. However, the Failover Group listener FQDN acts as a DNS abstraction layer across instances. Because NCC Private Endpoint rules are provisioned against specific target Azure Resource IDs and FQDNs, the compute environment cannot automatically resolve or route the Failover Group listener FQDN through the instance's private endpoint.

• DNS Private Zone Resolution - Failover Group listeners require the private DNS zone (database.windows.net) to maintain proper records mapping both the primary SQL MI and the Failover Group listener to the appropriate private endpoint IP.
• NCC Private Endpoint Scope - An NCC rule targeting only the primary SQL MI resource ID does not inherently intercept and route traffic for the Failover Group listener FQDN unless the alias is explicitly accounted for in the DNS path and endpoint definition.
• Cross-Region Architecture - If the secondary SQL MI instance is located in a secondary region, private endpoint routing becomes more complex and typically requires dedicated NCC configurations for each region's endpoint. 

You can try below

• Configure Private DNS Zone Records - Ensure the Azure Private DNS Zone for database.windows.net contains A records (or CNAMEs) for both primary instance and Failover Group listener. Both must resolve to the respective private IP of the active SQL MI Private Endpoint within your network.
• Align NCC Private Endpoint Rules - Check whether the NCC Private Endpoint rule can be configured to include the Failover Group listener domain name alongside the SQL MI resource, ensuring the Databricks Serverless DNS proxy correctly directs traffic across the private link.
• Direct Primary FQDN (Fallback) - If DNS routing for the listener cannot be resolved through NCC in the current setup, connecting directly to the primary SQL MI FQDN will maintain connectivity, though failover events will require updating the connection target.

[Samiko_db]

Hi @LokeshChikuru in your scenario, the private endpoint registered within the NCC is mapped to a single instance not the failover cluster, hence it fails. Currently Databricks serverless does not support DNS chasing or DNS redirects for resources accessed, this has been documented clearly under Microsoft own documentation here 

It states that:

• DNS chasing and DNS redirect are not supported. All domain names must resolve directly to the backend resources.

There is a potential workaround:

Databricks Serverless -> NCC Private Endpoint  -> Private Link Service -> Internal Standard load balancer -> proxy VM/ VM scale set -> SQL MI failover-group cluster 

Register a customer FQDN such as sqlmi-fg.contoso.com in the NCC and point it directly at the load-balancer/Private Link path. Do not use a CNAME chain to the SQL MI listener, and avoid private-use suffixes such as .internal, which the Databricks documentation excludes.

Test it and let me know.

[LokeshChikuru]

@Samiko_db 

Thanks for your response. I had a similar thought and noticed the same recommendation in a few other blogs as well. However, I recently came across the new Private Network Gateway feature for Azure Databricks, which leverages a delegated subnet and is currently in Private Preview.

Based on the documentation, it appears that this feature could potentially address DNS resolution challenges while also enabling access to specified endpoints outside of Azure. It may help resolve not only the current issue but some of the other networking constraints we have been encountering.

I am planning to test this feature in our non-production environment and evaluate whether it provides a viable solution before considering any further adoption.

Reference: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/priv...

[Samiko_db]

@LokeshChikuru 

Thank you for your message, Private Network Gateway (PNG) would assist you in creating a unified access to your cloud and on premise hosted resources without using various load balancers and proxy VM's. For the issue you are experiencing around Azure SQL MI, it is stated in our documentation that PNG will only connect to resources hosted in your virtual network, so it wont connect to cloud-hosted services such as ADLS and Azure SQL MI that uses service endpoints.

I would definitely encourage you to test PNG while its free in Private Preview.

Thanks
Sam