The Lakeflow connect Gateway setup, do we need to install the agent on-prem?

[Ashash12]

The Lakeflow connect Gateway setup to connect on-prem SQL server

Please provide the steps the setup the gateway agent on the on-prem? 

 Where to download this agent? What are the firewall rules for outbound looks like. 

Kind regards,

Asha

[szymon_dybczak · Accepted solution]

Hi @Ashash12 ,

You need to have proper network connectivity to your on premise SQL Server. 

As they stated in the docs - connector supports SQL Server on-premises using Azure ExpressRoute and AWS Direct Connect networking

https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/sql-server-pipeline

"The SQL Server connector supports Azure SQL Database, Azure SQL Managed Instance, and Amazon RDS SQL databases. This includes SQL Server running on Azure virtual machines (VMs) and Amazon EC2. The connector also supports SQL Server on-premises using Azure ExpressRoute and AWS Direct Connect networking"

[Ashash12]

Hi  @szymon_dybczak ,

Thank you for adding to my discussion.

Ingest data from SQL Server | Databricks on AWS

As per document, they *ALSO* Support via Azure ExpressRoute and AWS Direct Connect networking. Sounds like Ingestion Gateway needs this setup to go ahead when source and target are not in same network.  Setting up Azure ExpressRoute and AWS Direct Connect networking is a very costly activity just to consume a Single On-Prem Datasource. 

I was expecting a lightweight Gateway with an outbound connection to Databricks/Cloud should be the solution instead of ER/AWS Direct. 

This would be a suggestion to Databricks Innovation team. 

Kind Regards,

Asha Hiremath