Unity Catalog Enabled Clusters using PrivateNIC

newuser12445 — Wed, 24 Apr 2024 17:47:30 GMT

Hello,

When reviewing the VM settings for Databricks worker VMs, we can see that there are two(2) NICs.

A primary ( PublicNIC (primary)) and a secondary (PrivateNIC (primary)).

The workers VM is always assigned the PublicNIC and this is reachable from within the vNET.

When the Cluster access mode is set to "No isolation shared" the notebooks use the PublicNIC IP and can reach external resources.

We are unable to reach the PrivateNIC ip even within the same vNET.

When the Cluster access mode is set to "shared" (Unity Catalog enabled) the notebooks use the PrivateNIC IP and can NOT reach external resources

Would really appreciate some insight on this issue as Unity Catalog enabled Clusters us the PrivateNIC for all communications and therefore can't reach external resources.

topic Unity Catalog Enabled Clusters using PrivateNIC in Administration & Architecture

Unity Catalog Enabled Clusters using PrivateNIC