Serverless notebook DNS failure (gai error / name resolution)

bek04
New Contributor II

I’m using a Databricks workspace on AWS (region: us-west-2). My Serverless notebook (CPU) cannot access any external URL — every outbound request fails at DNS resolution.

Minimal test in a notebook:
import urllib.request
urllib.request.urlopen("https://www.google.com/robots.txt", timeout=20)

Errors I get:
- URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
- gaierror: [Errno -2] Name or service not known

Notes:
- SQL Warehouse works normally.
- Unity Catalog is enabled and I’m using Volumes (/Volumes/...).
- I already attached/updated the Network Connectivity Configuration (ncc_public_internet), but the notebook still has DNS failures.

What should I check next (network policy / serverless egress control / workspace setting) to enable outbound internet/DNS for Serverless notebooks?

emma_s
Databricks Employee
Databricks Employee

Hi,

Here are some troubleshooting steps:

1. Network Connectivity Configuration (NCC)

  • Confirm that the correct NCC (such as ncc_public_internet) is attached specifically to Serverless compute, not just to SQL Warehouses or other resources.
  • After making changes, allow several minutes for NCC propagation. Sometimes settings take time to apply.

2. Network Policies & Workspace Settings

  • Check whether workspace-level network policies restrict serverless notebook egress to the internet.
  • Review unified network policies (IAM roles, security groups, VPC endpoints) associated with your workspace for restrictions especially affecting serverless compute resources.
  • Ensure there are no workspace-level blocklists for outbound traffic, particularly for public DNS (UDP 53) or HTTP/HTTPS ports (TCP 80/443).

3. Serverless Egress Controls

  • Validate that serverless notebooks are allowed to make external outbound requests via public internet—some workspaces restrict egress for compliance.
  • Confirm that there are no workspace or admin settings that disable “Serverless Workspace egress to public internet.”
  • For Unity Catalog-enabled workspaces, verify egress settings within UC context, as newer workspace configurations sometimes require explicit enablement.

4. DNS Configuration

  • Some organizations use custom DNS resolvers or private DNS, which may block serverless resources from reaching the public DNS.
  • Check that public DNS resolution (e.g., Google’s 8.8.8.8, AWS’s default) is available, and that your workspace isn’t explicitly configured to use a resolver without internet access.

5. Workspace Isolation/Security Features

  • Inspect whether workspace “secure access” or “private link” features are enabled, as these may block all internet or DNS traffic unless explicitly permitted.
  • If PrivateLink or VPC-only network models are in use, ensure there are allowed egress rules.

6. Test Alternative Outbound Methods

  • As a diagnostic, attempt DNS resolution with different libraries (e.g., using socket.gethostbyname("www.google.com") or through ping) to determine if DNS works at all, or whether any outbound traffic is possible.
  • Try accessing other common endpoints besides Google (e.g., http://example.com)—sometimes specific domains are blocked but not all traffic.

bek04
New Contributor II

Hi, thanks for the troubleshooting checklist.

I tried the items above, but I’m still blocked and wanted to share the exact current state + ask what I should check next.

What I’ve done / verified:

  • Workspace is on AWS, region us-west-2.

  • Using Serverless notebook (CPU).

  • I attached the “ncc_public_internet” Network Connectivity Configuration to the workspace (I can see it on the workspace details page).

  • Serverless egress control shows the default policy with Access = Full.

  • After waiting and starting a fresh session, DNS still fails:

    • socket.gethostbyname for a public domain returns “gaierror: Name or service not known”

    • urllib request returns “Temporary failure in name resolution”

  • SQL Warehouse queries work, but any outbound HTTP call from the notebook fails because DNS can’t resolve.

Two extra clues:

  • When I open “Private access settings” in the account console, the page returns a 403 error (not sure if permissions-related or relevant).

  • In the account workspace overview, the credential name shows “Serverless only” (maybe this means classic clusters aren’t enabled?).

Question:
Given NCC is attached and egress policy is “Full”, what else can still block public DNS for Serverless notebooks? Is there a separate account-level setting/entitlement that must be enabled for serverless internet egress, or is “serverless-only” a known limitation where external downloads aren’t allowed?

Thanks again!

saurabh18cs
Honored Contributor II

Hi @bek04  can you please try once with direct ip address bypassing DNS, this will explain if egress is the issue or DNS. like : urllib.request.urlopen("https://142.250.74.206/robots.txt", timeout=20).read()[:100] ??

and 

nslookup www.google.com

and 

import socket
socket.getaddrinfo("www.google.com", 443)

Is you NCC VPC setup or workspace VPC either using custom DNS ? like enterprise level dns whether on-prem or custom dns then maybe good to check if  those servers forward public queries? becasue i believe workspace level policies like custom dns may influence how serverless works.