topic Re: Dynamic IP address in databricks in Data Engineering

Dynamic IP address in databricks

Tiwarisk — Tue, 19 Nov 2024 07:25:40 GMT

Everytime I am running a script in databricks which fetches data from a sql server(different Azure resource group) I am getting this error.

com.microsoft.sqlserver.jdbc.SQLServerException: Cannot open server 'proddatabase' requested by the login. Client with IP address '20.57.54.71' is not allowed to access the server. To enable access, use the Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect. ClientConnectionId:86ac86b7-9f9e-44cf-9b07-9172fb511aaf.

I have whitelisted the databricks vnet on the sql server but and given all the RBAC but still getting this.

Re: Dynamic IP address in databricks

ameet9257 — Tue, 19 Nov 2024 07:56:23 GMT

Hi @Tiwarisk

Have you whitelisted the entire address range or just one IP of VNET on the SQL Server?

Re: Dynamic IP address in databricks

Tiwarisk — Tue, 19 Nov 2024 08:26:00 GMT

Yes I whitelisted ip range.

Re: Dynamic IP address in databricks

Tiwarisk — Tue, 19 Nov 2024 08:27:25 GMT

The strange thing is that the ip in error is not even in the range of vnet ips.

Re: Dynamic IP address in databricks

ameet9257 — Tue, 19 Nov 2024 10:13:23 GMT

@Tiwarisk

Ideally, this should not happen because I also use a similar setup with Private VNET. So whenever the databricks cluster is getting up it gets the IPs within the VNET.

Please confirm if you have whitelisted the Private VNET IPs instead of the public ones.

Re: Dynamic IP address in databricks

Tiwarisk — Tue, 19 Nov 2024 10:55:25 GMT

I did the public one

Re: Dynamic IP address in databricks

ameet9257 — Tue, 19 Nov 2024 11:45:35 GMT

@Tiwarisk ,

If your Databricks is under the secure VNET then whitelist the Private VNET address range.