Connection reset error from Databricks notebook but works via curl (GCP)

[abhishek13]

Hi everyone,

I’m facing a connectivity issue in my Databricks workspace on GCP and would appreciate any guidance.

 Problem

When I run commands from a Databricks notebook, I see intermittent errors like:

Connection reset
Retrying request to https://us-east4.gcp.databricks.com:443

However, when I test connectivity manually from the cluster node using curl, it works fine.

 What I verified

1. Direct connectivity works

curl -v https://us-east4.gcp.databricks.com

• Resolves to public IP (34.x.x.x)
• TLS handshake successful
• Returns HTTP 303 → /login.html

1. DNS resolution is correct

getent hosts us-east4.gcp.databricks.com
→ 34.128.x.x

1. Proxy removed

• • Removed HTTP_PROXY / HTTPS_PROXY environment variables
  • Verified no proxy is being used

   Issue inside Databricks runtime

  • Notebook / Spark jobs still show:
    • Connection reset
    • Retry attempts in logs

  Questions

  1. Is this expected behavior due to connection reuse / keep-alive in Databricks runtime?
  2. Could this be related to JVM/Spark HTTP client behavior?
  3. Are there recommended configurations to avoid these connection reset logs?
  4. When should this be considered a real failure vs harmless retry?