topic Re: Need access to browse onprem SQL data in Data Engineering

Need access to browse onprem SQL data

BrianLind — Mon, 05 May 2025 13:42:29 GMT

Our BI team has started using Databricks and would like to browse our local (onprem) SQL database servers from within Databricks. I'm not sure if that's even possible.

So far, I've set up Databricks Secure Cluster Connectivity (SCC), created a private DNS zone, and established peering with the network where the SQL Server 2022 instance is hosted. I've configured the SQL server to accept connections from "app.reg", added the service principal under "User Management", created an OAuth secret, and granted my account permission to use it.

When I test the connection from a notebook using a Python script and SQL user credentials, I can successfully view data. However, I’m not sure how meaningful that test is, since I’ve learned that Notebooks and SQL Warehouses operate differently. It seems that SQL Warehouses must run in "Pro" mode to support network peering.

At this point, I’m uncertain whether I’m on the right path.

Re: Need access to browse onprem SQL data

Renu_ — Mon, 05 May 2025 15:09:53 GMT

Hi, based on what you’ve shared, it seems you’ve already completed many of the necessary steps. Just a few things to double-check as you move forward:

SQL Warehouses used for BI tools need to run in Pro mode, not serverless, since only Pro or Classic warehouses support VNet peering and private networking.
Ensure your firewall rules and DNS settings are correctly configured so the warehouse can successfully reach your SQL Server.
Additionally, consider using Unity Catalog and Databricks secrets for secure authentication.

Re: Need access to browse onprem SQL data

BrianLind — Mon, 05 May 2025 19:55:56 GMT

I have confirmed that the SQL warehouse is running in Pro mode and the firewall is open. When testing access via a notebook, I receive responses using both the IP address and DNS.

I just attempted to configure a foreign catalog using a connection based on an app registration. The connection test was successful. But, when I access the catalog, I only see a list of schemas like 'db_accessadmin', 'db_backupoperator', etc., and no tables are visible.