topic Re: Using Lakehouse Federation for SQL Server with Serverless Compute in Data Engineering

Using Lakehouse Federation for SQL Server with Serverless Compute

ss_data_eng — Tue, 11 Feb 2025 20:38:47 GMT

Hi,

My team was able to create a Foreign Catalog that connects to a SQL Server instance hosted on an Azure VM, however when trying to query the catalog, we cannot access it using serverless compute (or a serverless sql warehouse). We have tried looking into creating an NCC to connect, however there doesn't seem to be a way to create a Private Link to an Azure VM. How can we allow Databricks Serverless compute to access a SQL Server instance in an Azure VM?

Re: Using Lakehouse Federation for SQL Server with Serverless Compute

Alberto_Umana — Tue, 11 Feb 2025 21:41:20 GMT

I think you might have to setup VNet peering between the VNet where your SQL Server VM is located and the VNet used by your Databricks workspace with the corresponding NSGs with SQL server port.

Re: Using Lakehouse Federation for SQL Server with Serverless Compute

Rjdudley — Tue, 11 Feb 2025 21:42:59 GMT

Have you looked at Lakehouse Federation? What is Lakehouse Federation? - Azure Databricks | Microsoft Learn

Unless you've done something unusual, this is the reason Lakehouse Federation was invented.

Re: Using Lakehouse Federation for SQL Server with Serverless Compute

Akshay_Petkar — Wed, 12 Feb 2025 03:05:07 GMT

Hi @ss_data_eng ,
When copying the host, make sure it is correct. The host will be in URL format, typically starting with https:// . When copying the host, make sure it is correct. You need to copy the username, password, and host while removing jdbc:// or https:// from the host..

Re: Using Lakehouse Federation for SQL Server with Serverless Compute

Ralf — Tue, 23 Dec 2025 08:24:25 GMT

I'm trying to get something similar to work: Lakehouse Federation for Oracle with SQL warehouse serverless. We are using Azure Databricks and our Oracle DB runs on-prem. I've been able to use classic compute to query the database, but now I'd like to use SQL warehouse serverless compute, to be able to offer a fully virtualized set-up to clients. The Oracle database is not accessible from the internet.
What can I do to make this work (if it is a supported set-up)?

thanks,
Ralf