Open firewall to Azure Databricks workspace from AWS RDS machine/EC2 machine

[MaheshDR]

Hi All,

As part of our solution approach, we need to connect to one of our AWS RDS Oracle databases from Azure Databricks notebook.

We need your help to understand which IP range of Azure Databricks to consider to whitelist them on AWS RDS security group, so that we can access AWS RDS Oracle DB from our Azure Databricks notebook.

Please advise if we need perform any additional steps on Azure/Databricks side.

Thanks,

Mahesh

[Anonymous]

@Mahesh D​ :

To connect to an AWS RDS Oracle database from Azure Databricks, you need to whitelist the Azure Databricks IP range in the AWS RDS security group.

The IP range of Azure Databricks varies based on the region and deployment mode. You can find the IP range of Azure Databricks in your region and deployment mode from the the documentation.

Once you have the IP range of Azure Databricks, add it to the AWS RDS security group as an inbound rule for the Oracle database. You can refer to the following AWS documentation for more details on how to add inbound rules to security groups: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#AddRemoveRules

On the Azure Databricks side, you need to ensure that you have the necessary JDBC driver to connect to the Oracle database. You can download the JDBC driver from the Oracle website and upload it to Azure Databricks. Once you have the JDBC driver, you can use the standard JDBC connection method to connect to the Oracle database from Azure Databricks.

[Anonymous]

Hi @Mahesh D​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[MaheshDR]

Hi @Suteja Kanuri​ & @Vidula Khanna​ ,

Thanks for the info and the follow up.

I'm clear about the part "IP range of Azure Databricks in your region". I could find that the below IP range is meant for Australia East region where our Databricks workspace is created. We are pretty clear on what to do on AWS but we need more help from Azure perspective as we are new to it.

Australia East

Webapp - 13.75.218.172/32

SCC relay (if SCC is enabled) - tunnel.australiaeast.azuredatabricks.net

Control Plane NAT (if SCC is disabled) - 13.70.105.50/32

Extended infrastructure - 20.53.145.128/28

But I'm unsure of what is the deployment mode of our Databricks workspace.

Could you kindly advise how can I find out what is the deployment mode for our databricks workspace? And also, which IP range in specific from above we need to whitelist on our AWS RDS security groups?

Appreciate your help.

Kind regards,

Mahesh Donthireddy

[Anonymous]

@Mahesh D​ :

You can find the deployment mode of your Databricks workspace in the Azure portal.

1. Go to the Azure portal and navigate to your Databricks workspace.
2. Under the "Settings" section in the left-hand menu, click on "Deployment".
3. In the "Deployment" page, you should see the deployment mode listed under the "Summary" section.

Once you know the deployment mode, you can determine which IP range to whitelist on your AWS RDS security group.

If your workspace is deployed using Azure Databricks Standard SKU, you will need to whitelist the IP range for the "Control Plane NAT" listed in your region's IP range list. If your workspace is deployed using Azure Databricks Premium SKU, you will need to whitelist the IP range for the "Webapp" listed in your region's IP range list.

In your case, since your Databricks workspace is deployed in Australia East, you will need to whitelist the IP range "13.70.105.50/32" on your AWS RDS security group if your workspace is deployed using Azure Databricks Standard SKU. If your workspace is deployed using Azure Databricks Premium SKU, you will need to whitelist the IP range "13.75.218.172/32" on your AWS RDS security group.

[MaheshDR]

Hi @Suteja Kanuri​ ,

We identified that it is deployed as a premium SKU and tried whitelisting both IP ranges on our AWS RDS Security Group but we are still unable to connect from Databricks to RDS.

Any other thoughts?

Kind regards,

Mahesh

[Anonymous]

@Mahesh D​ :

If you have already whitelisted the Azure Databricks IP ranges on your AWS RDS security group and are still unable to connect, there may be some additional steps you can take.

First, ensure that you have configured the Oracle JDBC driver correctly in Azure Databricks.

Next, check if your RDS instance has publicly accessible enabled. If it is not, you may need to enable it to allow traffic from Azure Databricks to reach your RDS instance.

Also, ensure that the Oracle listener on your RDS instance is running and listening on the correct port. You can check the listener status and port number by connecting to the RDS instance using a tool like SQL*Plus or SQL Developer.

If you still cannot connect after trying these steps, you may want to check the network configuration of your VPC and subnets in AWS to ensure that there are no network security groups or ACLs that are blocking the traffic.

Lastly, you can enable VPC flow logs in your VPC to troubleshoot connectivity issues between your RDS instance and Azure Databricks. Flow logs can provide detailed information on the traffic flows and can help you identify any issues with the network configuration.