Call an Azure Function App with Access Restrictions from a Databricks Workspace

[erigaud]

Hello,

As the title says, I am trying to call an function from an Azure Function App configured with access restrictions from a python notebook in my Databricks workspace. The Function App resource is in a different subscription as the Databricks workspace.

I should point out that when I disable the access restrictions from the function app and enable the connection from all networks, the call from Databricks works fine. This suggests that there are missing rules in the restrictions, but I do not know which one to add. Also note that our workspace has Secure Cluster Connectivity enabled, meaning we do not have a public IP in the managed resource group of our workspace.

I tried adding our cluster IP address to the network rules of the function app, but this does not seem to work. 

Do you have an id of what rules/ip ranges should be added to the access restriction rules of the function app in order for this to work ? 

Thank you very much

[Kaniz]

Hi @erigaud,Configuring access restrictions for an Azure Function App can be a bit tricky, especially when you’re dealing with cross-subscription scenarios and Databricks workspaces.

Let’s break down the steps to help you achieve this:

1. Function App Access Restrictions:

   • First, ensure that your Azure Function App has the necessary access restrictions in place. Since you’ve mentioned that it works when you disable the restrictions, we need to identify which rules are missing.
   • Access restrictions are typically defined using IP whitelisting or virtual network service endpoints. Since your Databricks workspace doesn’t have a public IP, we’ll focus on IP whitelisting.
   • You mentioned adding your cluster IP address to the network rules, but it didn’t work. Let’s explore other options.

2. Identify the Necessary IP Ranges:

   • To allow communication between your Databricks workspace and the Function App, you’ll need to identify the IP ranges used by Databricks.
   • Unfortunately, Databricks doesn’t provide a fixed set of IP addresses because it dynamically allocates them based on the underlying infrastructure. However, there are some approaches you can try:

3. Dynamic IP Ranges:

   • Databricks provides a list of dynamic IP ranges that you can use for whitelisting. These ranges cover the IP addresses used by Databricks clusters, jobs, and other services.
   • You can find the list of dynamic IP ranges in the Databricks documentation.
   • Whitelist these IP ranges in your Function App’s access restrictions.

4. Shared Access Mode (Cluster ACLs):

   • In your Databricks workspace, create a cluster with Shared Access mode enabled. This mode restricts access to specific users or groups.
   • You can then grant access to this cluster (or a policy) using Cluster ACLs. This ensures that only authorized users can run SQL or Python within notebooks.
   • Use SQL GRANT statements to manage permissions for specific databases and tables.

5. Databricks REST API:

   • If you want to trigger Databricks jobs from your Function App, consider using the Databricks REST API.
   • You can create a job inside Databricks that uses your notebook, and then use the run-now REST endpoi...¹.
   • Good luck! 🚀

[erigaud]

Hello @Kaniz,

You mention that :

• You can find the list of dynamic IP ranges in the Databricks documentation.

Do you have a link to that point of the documentation ? I have not been able to find it. Thank you !

[erigaud · Accepted solution]

Update : 
Problem was fixed ! 
The key was to set an VNET rule in the access restriction, giving access directly to the subnets used by Databricks.

It seems like for Microsoft to Microsoft connections, the IP addresses are not used, so adding the IP ranges in the rule does nothing, but adding the subnet directly works.