cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

IP address fix

Jon
New Contributor II

How can I fix the IP address of my Azure Cluster so that I can whitelist the IP address to run my job daily on my python notebook? Or can I find out the IP address to perform whitelisting? Thanks

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III
  1. You can use pool for instances and set that there is "min idle" = "max capacity" so it will not auto terminate so ip will remain the same but this solution doesn't guarantee that it will not change (for example during upgrade or something)
  2. Preferred solution is when you do VM injection you can set routing and set that traffic through internet is going through firewall which have constant ip address. There is ready Azure firewall but a bit expensive, you can install own firewall on VM (there are ready images on Marketplace).
  3. Some other solution like connection through VPN or proxy requests are possible in some cases as alternative.

-werners-
Esteemed Contributor III

I would not go for option 1 indeed as Hubert said.

Azure provided solutions (VNET injection etc) are preferred.

The IP range which is used for the Azure VMs (which Databricks uses) will be quite big so whitelisting that whole range defeats the use of a firewall.

@-werners- Do you think there is any way we can use one single ip to access our target resource from databricks? I know databricks creates/uses so many ip's which is not feasible solution to whitelist each time. Please let me know if you have any idea on it. Thanks.

-werners-
Esteemed Contributor III

Depends on the scenario.  You could expose a single ip address to the external internet, but databricks itself will always use many addresses.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group