Azure Databricks Power Platform Connector - Doubts

JMartins777
New Contributor

‎07-01-2025 02:26 AM

Hello, 

Regarding the recently released azure databricks connector, i want to connect it to a Power App but i have 2 main questions which i need to know
1 - If the databricks URL is in a private network, how does it work and how can i achieve this connection.

2 - When i call this, it says it is real-time, but does it always call a cluster? How is it done, so i can get the data real-time without having to wait some minutes for the cluster to start?

Thanks in advance

0 Kudos

Louis_Frolio
Databricks Employee
Databricks Employee

‎07-01-2025 05:42 AM

Connecting Power Apps to Azure Databricks in Private Networks
  • If your Azure Databricks workspace uses private networking (like VNet injection or Private Link), Power Apps cannot connect to it by default due to restrictive network access.
  • To allow Power Apps to reach your Databricks endpoint:
    • You need to set up VNet peering or Private Link between the Databricks virtual network and the one that Power Platform uses.
    • Make sure authentication is done using Microsoft Entra ID—either OAuth or a service principal.
    • Once network connectivity is enabled and the right authentication is configured, you can add the Databricks connection in Power Apps.
  • If further restrictions exist (e.g., firewalls), you may need special components like an Azure Data Gateway, although this is more commonly documented for Power BI than Power Apps.
Real-Time Data Access and SQL Warehouse Behavior
  • The connector provides real-time data access by running queries on a Databricks SQL Warehouse, not by launching dedicated Spark clusters for each query.
  • If the SQL Warehouse is already running, results are returned quickly (seconds).
  • If the SQL Warehouse is paused or idle, the initial query after idling can take several minutes due to startup time. After it’s running, latency is minimal for subsequent queries.
  • You can minimize these delays by keeping the SQL Warehouse always running (using configuration to disable or extend auto-pause), but this increases compute cost.
Key Takeaways
  • For private networks: Ensure proper VNet peering/Private Link and use Entra ID authentication.
  • For real-time: All queries execute via SQL Warehouse; no Spark clusters are started. Keeping the SQL Warehouse active avoids cold-start delays.

Hope this helps, Lou.

View solution in original post

nayan_wylde
Esteemed Contributor II

‎07-01-2025 07:34 AM - edited ‎07-01-2025 07:39 AM

1. Private Network Connectivity

For Databricks workspaces in private networks, you have a couple of options:

Option A: On-premises Data Gateway

  • Install the Microsoft On-premises Data Gateway in your private network
  • The gateway acts as a bridge between Power Apps (cloud) and your private Databricks workspace
  • Configure the gateway to route traffic through your private endpoints
  • This is the most common approach for private connectivity scenarios

Option B: Private Endpoints + VNet Integration

Network Requirements:

  • Your private network must allow HTTPS (443) outbound to Power Platform services
  • Configure DNS to resolve Databricks private endpoint addresses
  • Ensure firewall rules allow the data gateway or VNet integration traffic


    2. Real-time Performance & Cluster Management

    The "real-time" capability is achieved through Databricks SQL Serverless, not traditional clusters

  • You can also try to put the cluster in some pool that you reduce cold start time.