Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2025 03:16 AM
Hey @Mendi , Here’s how connectivity works for Azure Databricks with VNet injection and Secure Cluster Connectivity (SCC) for the endpoints you listed.
Key points from the Microsoft Learn reference
- The page lists, per region, the FQDNs and ports for the workspace-level Hive metastore, artifact Blob storage, system tables storage, log Blob storage, and the Event Hubs endpoint that clusters must reach when you manage egress with UDRs/firewalls. IPs change, so allowlist the FQDNs and resolve IPs automatically if you must use IPs.
- Databricks advises using the Azure Databricks service tag rather than pinned IPs to avoid outages when addresses change.
1) Metastore connectivity: Databricks-managed Hive metastore vs external Hive metastore vs Unity Catalog
-
For the Databricks-managed Hive metastore, clusters need outbound access on port 3306/TCP to the regional MySQL FQDNs listed in the table (for West Europe:
consolidated-westeurope-prod-metastore.mysql.database.azure.comandconsolidated-westeurope-prod-metastore-addl-1.mysql.database.azure.com, among others). -
If you use an external Hive metastore, then your clusters must be able to reach the external metastore you configured (host + port) because the metastore client connects directly via JDBC in “local mode”; the external metastore doc shows configuring
javax.jdo.option.ConnectionURLto your MySQL host:port, which implies network connectivity from the cluster to that endpoint. -
When Unity Catalog is used, compute relies on system tables storage (the
dfs.core.windows.netendpoint) over HTTPS (443) and not on the MySQL-based Hive metastore endpoints; the MySQL endpoints in the page are for the legacy Hive metastore, while UC’s workspace-level system tables storage is explicitly listed in the table as HTTPS-only. -
Azure’s VNet-injection guidance also notes that outbound NSG rules must permit ports 443, 3306, and 8443–8451 for service operation in injected workspaces; 3306 is specifically relevant to Hive metastore connectivity (managed or external), not Unity Catalog by itself.
West Europe examples (from the table)
- Metastore (MySQL, 3306/TCP):
consolidated-westeurope-prod-metastore.mysql.database.azure.com,consolidated-westeurope-prod-metastore-addl-1.mysql.database.azure.com(and additional listed FQDNs) -
System tables storage (HTTPS, 443):
ucstprdwesteu.dfs.core.windows.net -
Artifact storage (HTTPS, 443):
dbartifactsprodwesteu.blob.core.windows.net(plus additional artifact endpoints) -
Log Blob storage (HTTPS, 443):
dblogprodwesteurope.blob.core.windows.net
2) Event Hubs endpoints on 9093/TCP: required and why
-
The page lists a regional Event Hubs endpoint FQDN (for West Europe:
prod-westeurope-observabilityeventhubs.servicebus.windows.net) with port 9093/TCP; if you control egress via UDRs/firewalls, you should allow outbound to this FQDN and port so Databricks services can operate correctly in VNet-injected/SCC environments. -
Port 9093 is the Kafka-compatible port exposed by Azure Event Hubs, which Databricks uses with Kafka clients; official Databricks instructions for Event Hubs/Kafka explicitly configure
bootstrap.serverswith:9093and describe 9093 as the Event Hubs Kafka port. -
The endpoint naming includes “observabilityeventhubs,” and Databricks documents and examples show Event Hubs used for streaming/telemetry via Kafka-compatible endpoints, which is why the platform publishes this endpoint and port in the regional table.
Practical guidance
- Prefer the Azure Databricks service tag in NSGs/UDRs; if your policy requires explicit FQDN/IP allowlisting, use the table’s FQDNs and resolve IPs periodically because these can change.
-
If you’re on Unity Catalog and don’t use the legacy Hive metastore, ensure HTTPS (443) to the region’s UC system tables storage and the artifact/log storage FQDNs; you don’t need the MySQL 3306 endpoints for UC-only metadata paths.
-
If you use the Databricks-managed Hive metastore, open 3306/TCP to the regional MySQL metastore FQDNs listed; same applies for an external Hive metastore—open to your external metastore host:port and follow the external metastore configuration doc.
-
Keep Event Hubs 9093/TCP open to the regional “observabilityeventhubs” FQDN to ensure Kafka-compatible Event Hubs interactions and platform observability paths function.
Cheers, Louis.