topic IP address for accessing external SFTP server in Data Engineering

IP address for accessing external SFTP server

ande — Fri, 26 Apr 2024 14:48:50 GMT

I am trying to pull in data to my Databricks workspace via an external SFTP server. I am using Azure for my compute. To access the SFTP server they need to whitelist my IP address. My IP address in Azure Databricks seems to be constantly changing from day to day.

What is the solution to get around this issue?

Re: IP address for accessing external SFTP server

Walter_C — Sat, 27 Apr 2024 16:08:54 GMT

Azure Databricks, like many cloud services, does not provide static IP addresses for outbound connections. This is because the compute resources are dynamically allocated and can change over time.

One potential workaround could be to use a Virtual Network (VNet) with a Network Address Translation (NAT) gateway. The NAT gateway can provide a stable outbound IP address. However, this would require additional setup and configuration in Azure, and may have additional costs.

Re: IP address for accessing external SFTP server

jeremy98 — Wed, 23 Jul 2025 16:24:50 GMT

Hi,

My team and I set up a NAT gateway this morning to ensure a stable IP address. However, we still don’t have authentication to send data to the external SFTP server.

I tried creating an outbound rule in the Network Security Group associated with the Databricks workspace, but it's still not working.

Could you please let me know if I’ve configured it correctly? If not, I’d really appreciate your guidance — I’m not very experienced with setting up these resources.

Thanks in advance!