topic Re: Serverless notebook DNS failure (gai error / name resolution) in Data Engineering

Serverless notebook DNS failure (gai error / name resolution)

bek04 — Tue, 23 Dec 2025 01:49:38 GMT

I’m using a Databricks workspace on AWS (region: us-west-2). My Serverless notebook (CPU) cannot access any external URL — every outbound request fails at DNS resolution.

Minimal test in a notebook:
import urllib.request
urllib.request.urlopen("https://www.google.com/robots.txt", timeout=20)

Errors I get:
- URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
- gaierror: [Errno -2] Name or service not known

Notes:
- SQL Warehouse works normally.
- Unity Catalog is enabled and I’m using Volumes (/Volumes/...).
- I already attached/updated the Network Connectivity Configuration (ncc_public_internet), but the notebook still has DNS failures.

What should I check next (network policy / serverless egress control / workspace setting) to enable outbound internet/DNS for Serverless notebooks?

Re: Serverless notebook DNS failure (gai error / name resolution)

emma_s — Tue, 23 Dec 2025 15:31:04 GMT

Hi,

Here are some troubleshooting steps:

1. Network Connectivity Configuration (NCC)

Confirm that the correct NCC (such as ncc_public_internet) is attached specifically to Serverless compute, not just to SQL Warehouses or other resources.
After making changes, allow several minutes for NCC propagation. Sometimes settings take time to apply.

2. Network Policies & Workspace Settings

Check whether workspace-level network policies restrict serverless notebook egress to the internet.
Review unified network policies (IAM roles, security groups, VPC endpoints) associated with your workspace for restrictions especially affecting serverless compute resources.
Ensure there are no workspace-level blocklists for outbound traffic, particularly for public DNS (UDP 53) or HTTP/HTTPS ports (TCP 80/443).

3. Serverless Egress Controls

Validate that serverless notebooks are allowed to make external outbound requests via public internet—some workspaces restrict egress for compliance.
Confirm that there are no workspace or admin settings that disable “Serverless Workspace egress to public internet.”
For Unity Catalog-enabled workspaces, verify egress settings within UC context, as newer workspace configurations sometimes require explicit enablement.

4. DNS Configuration

Some organizations use custom DNS resolvers or private DNS, which may block serverless resources from reaching the public DNS.
Check that public DNS resolution (e.g., Google’s 8.8.8.8, AWS’s default) is available, and that your workspace isn’t explicitly configured to use a resolver without internet access.

5. Workspace Isolation/Security Features

Inspect whether workspace “secure access” or “private link” features are enabled, as these may block all internet or DNS traffic unless explicitly permitted.
If PrivateLink or VPC-only network models are in use, ensure there are allowed egress rules.

6. Test Alternative Outbound Methods

As a diagnostic, attempt DNS resolution with different libraries (e.g., using socket.gethostbyname("www.google.com") or through ping) to determine if DNS works at all, or whether any outbound traffic is possible.
Try accessing other common endpoints besides Google (e.g., http://example.com)—sometimes specific domains are blocked but not all traffic.

Re: Serverless notebook DNS failure (gai error / name resolution)

bek04 — Tue, 23 Dec 2025 20:55:12 GMT

Hi, thanks for the troubleshooting checklist.

I tried the items above, but I’m still blocked and wanted to share the exact current state + ask what I should check next.

What I’ve done / verified:

Workspace is on AWS, region us-west-2.
Using Serverless notebook (CPU).
I attached the “ncc_public_internet” Network Connectivity Configuration to the workspace (I can see it on the workspace details page).
Serverless egress control shows the default policy with Access = Full.
After waiting and starting a fresh session, DNS still fails:
- socket.gethostbyname for a public domain returns “gaierror: Name or service not known”
- urllib request returns “Temporary failure in name resolution”
SQL Warehouse queries work, but any outbound HTTP call from the notebook fails because DNS can’t resolve.

Two extra clues:

When I open “Private access settings” in the account console, the page returns a 403 error (not sure if permissions-related or relevant).
In the account workspace overview, the credential name shows “Serverless only” (maybe this means classic clusters aren’t enabled?).

Question:
Given NCC is attached and egress policy is “Full”, what else can still block public DNS for Serverless notebooks? Is there a separate account-level setting/entitlement that must be enabled for serverless internet egress, or is “serverless-only” a known limitation where external downloads aren’t allowed?

Thanks again!

Re: Serverless notebook DNS failure (gai error / name resolution)

saurabh18cs — Wed, 24 Dec 2025 09:35:35 GMT

Hi @bek04 can you please try once with direct ip address bypassing DNS, this will explain if egress is the issue or DNS. like : urllib.request.urlopen("https://142.250.74.206/robots.txt", timeout=20).read()[:100] ??

and

nslookup www.google.com

and

import socket

socket.getaddrinfo("www.google.com", 443)

Is you NCC VPC setup or workspace VPC either using custom DNS ? like enterprise level dns whether on-prem or custom dns then maybe good to check if those servers forward public queries? becasue i believe workspace level policies like custom dns may influence how serverless works.