topic Re: Failed to resolve External API in Data Engineering

Failed to resolve External API

somnii — Tue, 15 Sep 2026 04:29:37 GMT

I am currently testing on a Premium trial, but my workspace is unable to resolve my external API to pull data. May I know if I need to update anything on the settings side to fix this issue?

Thanks!

Re: Failed to resolve External API

Satyasai — Tue, 15 Sep 2026 05:06:57 GMT

Hi @somnii
If You Are Running on Serverless Compute

Serverless workloads run within the Databricks-managed control plane. Outbound internet calls from Serverless Compute are governed by Serverless Egress Policies at the Account level:

Restricted Egress Mode: By default, serverless network policies for many accounts are in a restricted mode to prevent arbitrary outbound internet access.

How to Fix (Account Admin Required):

Log into the Databricks Account Console (accounts.cloud.databricks.com or platform equivalent).

Navigate to Security > Networking > Context-based ingress & egress control

Locate the policy associated with your trial workspace

On the Egress tab, ensure Internet access is set to "Allow access to all destinations" or explicitly add your external API domain (FQDN) to the Allowed Destinations list.

Restart your serverless session and re-run your request.

2. If You Are Running on Classic Compute (Single Node / Multi-Node Clusters)

Classic clusters run in the cloud virtual networks (VPC/VNet). Trial workspaces often deploy into a Databricks-managed Default VPC/VNet:

Egress Restrictions / Firewall Rules: If your trial workspace was provisioned using a Customer-Managed VPC (VNet Injection) then outbound internet traffic is governed by your cloud networks' Security Groups, NAT Gateway, or Egress Firewall.

IP Whitelisting on the External API: If the API endpoint you are calling is enforcing IP whitelisting then calls made from a Databricks cluster would originate from the public IP of your NAT Gateway or Public Node IP. You will need to whitelist these public egress IPs on the external API side.

PySpark UDF Internet Access Restriction: If your API request is wrapped in a PySpark User-Defined Function (UDF) spanning across worker nodes, be sure node-to-node security group rules allow outbound 80/443 traffic.

Quick Troubleshooting Verification

Try executing a simple Python request directly from a notebook attached to a Classic All-Purpose Cluster:

Python

import requests

try:

response = requests.get("https://api.your-endpoint.com/health", timeout=10)

print("Status:", response.status_code)

except Exception as e:

print("Connection Failed:", e)

If Classic Compute succeeds but Serverless fails: This indicates a Serverless Egress Policy restriction in the Account Console.

If both fail with a DNS/Timeout error: This indicates a cloud network firewall, proxy configuration, or the target API is blocking Databricks egress IPs.

Re: Failed to resolve External API

data_pulse — Tue, 15 Sep 2026 06:37:40 GMT

Hi @somnii

First check whether the notebook is running on Serverless or Classic compute, because the troubleshooting path is different.

A quick test can separate DNS resolution from HTTPS connectivity, replace your host and try this:

import socket, requests host = "api.example.com" try: print("DNS:", socket.gethostbyname(host)) except Exception as e: print("DNS failed:", e) try: print("HTTPS:", requests.get(f"https://{host}", timeout=10).status_code) except Exception as e: print("HTTPS failed:", e)

You can also run:

%sh curl -I -v https://api.example.com

curl -v is useful because it can quickly show whether the failure is at DNS resolution, TCP connection, TLS handshake, or HTTP/API layer.

If DNS fails → focus on name resolution / outbound networking.
If DNS works but HTTPS fails → check firewall/proxy, IP allowlisting, TLS, or API endpoint itself.

For Serverless, also check:

SELECT event_time, event_id, destination, access_type, dns_event
FROM system.access.outbound_network
WHERE event_time >= current_timestamp() - INTERVAL 30 MINUTES
ORDER BY event_time DESC

If the destination shows access_type = 'DROP', that points to the serverless outbound network path.

For Classic compute, check the workspace VPC/VNet egress path instead: NAT/internet access, firewall/security-group rules, proxy settings, and whether the external API requires source-IP allowlist. Databricks docs the required subnet routing, NAT gateway, security-group, and outbound-access configuration here

If Serverless shows access_type = 'DROP', check whether your account level exposes any serverless egress/network policy controls. If no relevant setting is available and the issue is only on Serverless, then Databricks Support may need to investigate further.

Re: Failed to resolve External API

mancy34 — Tue, 15 Sep 2026 07:32:15 GMT

This is usually related to network access rather than the Premium trial itself. I’d check the workspace’s serverless/external network access settings and make sure the API domain is allowed, then verify DNS resolution and any firewall or IP allowlist on the API side. If it still fails, the exact error from the workspace logs should point to whether it’s DNS, authentication, or outbound connectivity.

Re: Failed to resolve External API

anuj_lathi — Fri, 09 Oct 2026 11:48:27 GMT

This is almost always an outbound networking problem, not a setting you toggle on the workspace itself. Which fix applies depends on what compute your notebook is running on.

1. Confirm it's DNS or egress, and not the API itself

Run this from the same compute that's failing:

import socket, requests

host = "api.yourservice.com"   # replace with your API host
print(socket.getaddrinfo(host, 443))          # fails here = DNS resolution problem
print(requests.get(f"https://{host}", timeout=10).status_code)  # fails here = connectivity or blocking

A Name or service not known error from getaddrinfo means DNS is failing or being blocked.
A timeout or connection refused after DNS succeeds points to a firewall, security group, or route problem.

2. If you're on serverless compute

Serverless runs in the Databricks-managed serverless compute plane, and outbound access is governed by serverless egress controls and network policies [1]. If a policy restricts outbound traffic, calls to any destination you haven't explicitly allowed will fail, and that often shows up as a resolution error.

What to check:

Ask an account admin whether a network policy with restricted egress is attached to your workspace.
If it is, add your API's domain to the allowed destinations, or use a policy that permits the access you need.
If your API provider only accepts traffic from allow-listed IPs, you'll need the stable IPs for serverless compute so you can configure their firewall [1].

See What is serverless egress control?, Manage network policies for serverless egress control, and Configure a firewall for serverless compute access in the networking docs [1].

3. If you're on classic compute (clusters in your own VPC)

If your workspace uses a customer-managed VPC, outbound traffic from the cluster depends on your AWS networking, not on a Databricks setting [1]. Check:

DNS: the VPC has DNS resolution and DNS hostnames enabled, and any custom DHCP/DNS settings can resolve public names.
Routing: private subnets have a route to a NAT gateway (or other egress path) for 0.0.0.0/0.
Security groups and NACLs: outbound rules allow HTTPS (443) to your API.
Firewall or proxy: if you route egress through one, your API's domain needs to be allowed.

See Deploy Databricks in a customer-managed VPC and Classic compute plane networking [1].

4. If the API is private

If the API sits inside your own VPC or behind private endpoints, public DNS won't resolve it from Databricks. You'd need private connectivity (VPC peering for classic compute, or private endpoint rules for serverless) [1].

Next step

Tell me which compute type you're using (serverless or a classic cluster) and the exact error text from the test above. For a standard trial with no custom networking, serverless egress policy is the first thing I'd check. If you set up a customer-managed VPC, check its DNS and NAT first.

References

[1] Networking | Databricks on AWS — https://docs.databricks.com/aws/en/security/network