Network configuration doesn’t have to be hard! In this article, I will walk you through different scenarios of network configuration for accessing Azure Key Vault(AKV) from Databricks.
AKV backed Databricks Secrets Scopes
Network configuration is straightforward in this scenario
Accessing AKV from Enhanced Python UDFs in Unity Catalog
In this article, I showed you a step-by-step guide of how to strengthen Data Privacy with enhanced Python UDFs in Databricks Unity Catalog(UC).
The network configuration of accessing AKV from UC UDFs is different from using AKV for Databricks secrets scopes.
Use Private Endpoints for Maximum Security
- Create a Private Endpoint Connection for the AKV instance
- Create a DNS record to connect to your Private Endpoint
NOTE: Even though you can use your own DNS servers, it saves you a ton of headache to just use Azure Private DNS Zone
Use Firewalls and Virtual Networks
Alternative to Private Endpoint, you can enable secure communication between your Databricks Virtual Networks and AKV via Firewalls and Virtual Networks
That’s it! Now you can configure network access for your AKV and access AKV from Enhanced Python UDFs in Unity Catalog. Give it a try!
