Failed to resolve External API

[somnii]

I am currently testing on a Premium trial, but my workspace is unable to resolve my external API to pull data. May I know if I need to update anything on the settings side to fix this issue?

Thanks!

[Satyasai]

Hi @somnii 
If You Are Running on Serverless Compute

Serverless workloads run within the Databricks-managed control plane. Outbound internet calls from Serverless Compute are governed by Serverless Egress Policies at the Account level:

Restricted Egress Mode: By default, serverless network policies for many accounts are in a restricted mode to prevent arbitrary outbound internet access.

How to Fix (Account Admin Required):

Log into the Databricks Account Console (accounts.cloud.databricks.com or platform equivalent).

Navigate to Security > Networking > Context-based ingress & egress control

Locate the policy associated with your trial workspace

On the Egress tab, ensure Internet access is set to "Allow access to all destinations" or explicitly add your external API domain (FQDN) to the Allowed Destinations list.

Restart your serverless session and re-run your request.

2. If You Are Running on Classic Compute (Single Node / Multi-Node Clusters)

Classic clusters run in the cloud virtual networks (VPC/VNet). Trial workspaces often deploy into a Databricks-managed Default VPC/VNet:

Egress Restrictions / Firewall Rules: If your trial workspace was provisioned using a Customer-Managed VPC (VNet Injection) then outbound internet traffic is governed by your cloud networks' Security Groups, NAT Gateway, or Egress Firewall.

IP Whitelisting on the External API: If the API endpoint you are calling is enforcing IP whitelisting then calls made from a Databricks cluster would originate from the public IP of your NAT Gateway or Public Node IP. You will need to whitelist these public egress IPs on the external API side.

PySpark UDF Internet Access Restriction: If your API request is wrapped in a PySpark User-Defined Function (UDF) spanning across worker nodes, be sure node-to-node security group rules allow outbound 80/443 traffic.

Quick Troubleshooting Verification

Try executing a simple Python request directly from a notebook attached to a Classic All-Purpose Cluster:

Python

import requests

try:

response = requests.get("https://api.your-endpoint.com/health", timeout=10)

print("Status:", response.status_code)

except Exception as e:

print("Connection Failed:", e)

If Classic Compute succeeds but Serverless fails: This indicates a Serverless Egress Policy restriction in the Account Console.

If both fail with a DNS/Timeout error: This indicates a cloud network firewall, proxy configuration, or the target API is blocking Databricks egress IPs.

[data_pulse]

Hi @somnii 

First check whether the notebook is running on Serverless or Classic compute, because the troubleshooting path is different.

A quick test can separate DNS resolution from HTTPS connectivity, replace your host and try this:

import socket, requests
host = "api.example.com"

try:
    print("DNS:", socket.gethostbyname(host))
except Exception as e:
    print("DNS failed:", e)

try:
    print("HTTPS:", requests.get(f"https://{host}", timeout=10).status_code)
except Exception as e:
    print("HTTPS failed:", e)

You can also run:

%sh
curl -I -v https://api.example.com

curl -v is useful because it can quickly show whether the failure is at DNS resolution, TCP connection, TLS handshake, or HTTP/API layer.

If DNS fails → focus on name resolution / outbound networking.
If DNS works but HTTPS fails → check firewall/proxy, IP allowlisting, TLS, or API endpoint itself.

For Serverless, also check:

SELECT event_time, event_id, destination, access_type, dns_event
FROM system.access.outbound_network
WHERE event_time >= current_timestamp() - INTERVAL 30 MINUTES
ORDER BY event_time DESC

If the destination shows access_type = 'DROP', that points to the serverless outbound network path.

For Classic compute, check the workspace VPC/VNet egress path instead: NAT/internet access, firewall/security-group rules, proxy settings, and whether the external API requires source-IP allowlist. Databricks docs the required subnet routing, NAT gateway, security-group, and outbound-access configuration here

If Serverless shows access_type = 'DROP', check whether your account level exposes any serverless egress/network policy controls. If no relevant setting is available and the issue is only on Serverless, then Databricks Support may need to investigate further.

[mancy34]

This is usually related to network access rather than the Premium trial itself. I’d check the workspace’s serverless/external network access settings and make sure the API domain is allowed, then verify DNS resolution and any firewall or IP allowlist on the API side. If it still fails, the exact error from the workspace logs should point to whether it’s DNS, authentication, or outbound connectivity.