Not able to call external API when using Databricks free edition

AKB_10 — Sat, 26 Jul 2025 15:06:53 GMT

Hello guys , I'm new to Databricks and trying to create ETL pipelines for my personal projects . I'm using the Databricks free Edition but whenever I try to call an API to an external source to extract Data I always get this error .

I have attached the Image of the error below .

Why this keeps happening , am I not aware of something or doing something wrong ? Can someone please help ?

Re: Not able to call external API when using Databricks free edition

lingareddy_Alva — Sat, 26 Jul 2025 16:18:55 GMT

Hi @AKB_10

This is a common issue with Databricks Community Edition (free tier). Here's what's happening and how to fix it:
Root Cause
Databricks Community Edition has network restrictions that block outbound API calls to external services like Spotify,
social media APIs, etc. This is a security limitation of the free tier.
That’s why you get a DNS resolution error: the environment cannot resolve or connect to accounts.spotify.com.
This is a security and cost-control limitation. Since the free Community Edition runs on a shared infrastructure, Databricks blocks:
- Internet-bound traffic
- Some file system access
- Custom library installations from the internet

Solutions:

1. Upgrade to Databricks Premium or use a cloud-connected workspace
If you want to access external APIs:
- Use Databricks on AWS, Azure, or GCP with your own cloud subscription.
- Set up internet access through NAT gateway if on VPC.

2. Develop locally (e.g., in Jupyter or VS Code)
Run your ETL code on your local machine, where you have full internet access. Then:
- Upload results to Databricks if needed for further processing.

topic Not able to call external API when using Databricks free edition in Data Engineering

Not able to call external API when using Databricks free edition

Re: Not able to call external API when using Databricks free edition