Using Lakehouse Federation for SQL Server with Serverless Compute

[ss_data_eng]

Hi,

My team was able to create a Foreign Catalog that connects to a SQL Server instance hosted on an Azure VM, however when trying to query the catalog, we cannot access it using serverless compute (or a serverless sql warehouse). We have tried looking into creating an NCC to connect, however there doesn't seem to be a way to create a Private Link to an Azure VM. How can we allow Databricks Serverless compute to access a SQL Server instance in an Azure VM?

[Alberto_Umana]

I think you might have to setup VNet peering between the VNet where your SQL Server VM is located and the VNet used by your Databricks workspace with the corresponding NSGs with SQL server port.

[Rjdudley]

Have you looked at Lakehouse Federation? What is Lakehouse Federation? - Azure Databricks | Microsoft Learn

Unless you've done something unusual, this is the reason Lakehouse Federation was invented.

[Akshay_Petkar]

Hi @ss_data_eng ,
When copying the host, make sure it is correct. The host will be in URL format, typically starting with https:// . When copying the host, make sure it is correct. You need to copy the username, password, and host while removing jdbc:// or https:// from the host..