Serverless notebook DNS failure (gai error / name resolution)

[bek04]

I’m using a Databricks workspace on AWS (region: us-west-2). My Serverless notebook (CPU) cannot access any external URL — every outbound request fails at DNS resolution.

Minimal test in a notebook:
import urllib.request
urllib.request.urlopen("https://www.google.com/robots.txt", timeout=20)

Errors I get:
- URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
- gaierror: [Errno -2] Name or service not known

Notes:
- SQL Warehouse works normally.
- Unity Catalog is enabled and I’m using Volumes (/Volumes/...).
- I already attached/updated the Network Connectivity Configuration (ncc_public_internet), but the notebook still has DNS failures.

What should I check next (network policy / serverless egress control / workspace setting) to enable outbound internet/DNS for Serverless notebooks?

[emma_s]

Hi,

Here are some troubleshooting steps:

1. Network Connectivity Configuration (NCC)

• Confirm that the correct NCC (such as ncc_public_internet) is attached specifically to Serverless compute, not just to SQL Warehouses or other resources.
• After making changes, allow several minutes for NCC propagation. Sometimes settings take time to apply.

2. Network Policies & Workspace Settings

• Check whether workspace-level network policies restrict serverless notebook egress to the internet.
• Review unified network policies (IAM roles, security groups, VPC endpoints) associated with your workspace for restrictions especially affecting serverless compute resources.
• Ensure there are no workspace-level blocklists for outbound traffic, particularly for public DNS (UDP 53) or HTTP/HTTPS ports (TCP 80/443).

3. Serverless Egress Controls

• Validate that serverless notebooks are allowed to make external outbound requests via public internet—some workspaces restrict egress for compliance.
• Confirm that there are no workspace or admin settings that disable “Serverless Workspace egress to public internet.”
• For Unity Catalog-enabled workspaces, verify egress settings within UC context, as newer workspace configurations sometimes require explicit enablement.

4. DNS Configuration

• Some organizations use custom DNS resolvers or private DNS, which may block serverless resources from reaching the public DNS.
• Check that public DNS resolution (e.g., Google’s 8.8.8.8, AWS’s default) is available, and that your workspace isn’t explicitly configured to use a resolver without internet access.

5. Workspace Isolation/Security Features

• Inspect whether workspace “secure access” or “private link” features are enabled, as these may block all internet or DNS traffic unless explicitly permitted.
• If PrivateLink or VPC-only network models are in use, ensure there are allowed egress rules.

6. Test Alternative Outbound Methods

• As a diagnostic, attempt DNS resolution with different libraries (e.g., using socket.gethostbyname("www.google.com") or through ping) to determine if DNS works at all, or whether any outbound traffic is possible.
• Try accessing other common endpoints besides Google (e.g., http://example.com)—sometimes specific domains are blocked but not all traffic.

[bek04]

Hi, thanks for the troubleshooting checklist.

I tried the items above, but I’m still blocked and wanted to share the exact current state + ask what I should check next.

What I’ve done / verified:

• Workspace is on AWS, region us-west-2.

• Using Serverless notebook (CPU).

• I attached the “ncc_public_internet” Network Connectivity Configuration to the workspace (I can see it on the workspace details page).

• Serverless egress control shows the default policy with Access = Full.

• After waiting and starting a fresh session, DNS still fails:

  • socket.gethostbyname for a public domain returns “gaierror: Name or service not known”

  • urllib request returns “Temporary failure in name resolution”

• SQL Warehouse queries work, but any outbound HTTP call from the notebook fails because DNS can’t resolve.

Two extra clues:

• When I open “Private access settings” in the account console, the page returns a 403 error (not sure if permissions-related or relevant).

• In the account workspace overview, the credential name shows “Serverless only” (maybe this means classic clusters aren’t enabled?).

Question:
Given NCC is attached and egress policy is “Full”, what else can still block public DNS for Serverless notebooks? Is there a separate account-level setting/entitlement that must be enabled for serverless internet egress, or is “serverless-only” a known limitation where external downloads aren’t allowed?

Thanks again!

[saurabh18cs]

Hi @bek04  can you please try once with direct ip address bypassing DNS, this will explain if egress is the issue or DNS. like : urllib.request.urlopen("https://142.250.74.206/robots.txt", timeout=20).read()[:100] ??

and 

nslookup www.google.com

and 

import socket

socket.getaddrinfo("www.google.com", 443)

Is you NCC VPC setup or workspace VPC either using custom DNS ? like enterprise level dns whether on-prem or custom dns then maybe good to check if  those servers forward public queries? becasue i believe workspace level policies like custom dns may influence how serverless works.