topic Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve in Data Engineering

Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolves

sudopradip — Mon, 26 Oct 2026 10:15:10 GMT

From Azure Databricks serverless notebook compute (East US), uploading a file to a Unity Catalog volume with the Databricks SQL Connector PUT ... INTO '/Volumes/...' fails at TCP connect with [Errno 111] Connection refused. The warehouse accepts the PUT and returns a signed upload URL, but the client upload to the storage gateway fails.

I hit this via the open-source dlt (data load tool) Databricks destination, whose "direct load" mode stages files with this exact PUT. I've reproduced it with the bare connector and no dlt, in two separate workspaces.

Environment

Cloud/region: Azure, East US (sn=az-eastus-c3 in the signed URL)
Compute: serverless notebook, environment version [...]
databricks-sql-connector 4.8.0
dlt [...] (not needed for the repro)
Warehouse: serverless SQL warehouse
Metastore: East US Unity Catalog metastore

Network / egress configuration (checked in the account console)

Network connectivity configuration (NCC): None
Network policy: default-policy, egress "Allow access to all destinations"
Compliance security profile: Disabled
system.access.outbound_network: no rows at all for the account, so no denied events are logged

Error

HTTPSConnectionPool(host='eastus.storage.azuredatabricks.net', port=443): Max retries exceeded with url:
/api/2.0/fs/files/Volumes/CATALOG/SCHEMA/VOLUME/put_test.txt?...&X-Databricks-Consumer-Network-Zone=internet&sn=az-eastus-c3&...
(Caused by NewConnectionError(... [Errno 111] Connection refused))

What I found

Inside serverless, eastus.storage.azuredatabricks.net resolves to 192.168.200.28 (private), which refuses port 443.
Outside Databricks, the same hostname resolves to 52.234.168.242 (public).
From the same serverless compute, forcing the connection to 52.234.168.242 completes TLS and returns HTTP 400 (expected for an unsigned request), so the gateway itself is reachable and only the internal DNS answer is wrong.
The signed URL carries X-Databricks-Consumer-Network-Zone=internet, yet serverless routes that host to an internal IP.
From the same serverless compute, writing to the same volume works through:
- /Volumes/... FUSE (shutil.copy): ✅
- Databricks SDK WorkspaceClient().files.upload() (workspace host): ✅
- SQL connector PUT (storage gateway): ❌
Reproduced in a brand-new serverless workspace with default settings: the same private IP, the same error, the same az-eastus-c3.
The same PUT worked on classic compute in the original workspace.

Minimal repro (serverless notebook)

In serverless notebooks, the built-in databricks package shadows the connector's databricks.sql, so the path patch is needed:

%pip install databricks-sql-connector

dbutils.library.restartPython()

import sys, os, socket, databricks
for p in sys.path:
    c = os.path.join(p, "databricks")
    if os.path.isdir(os.path.join(c, "sql")) and c not in databricks.__path__:
        databricks.__path__.append(c)
from databricks import sql

print(socket.gethostbyname("eastus.storage.azuredatabricks.net"))  # -> 192.168.200.28

ctx = dbutils.notebook.entry_point.getDbutils().notebook().getContext()
open("/tmp/t.txt", "w").write("hi")
with sql.connect(server_hostname="<workspace-host>",
                 http_path="/sql/1.0/warehouses/<id>",
                 access_token=ctx.apiToken().get(),
                 staging_allowed_local_path="/tmp") as conn:
    conn.cursor().execute(
        "PUT '/tmp/t.txt' INTO '/Volumes/<catalog>/<schema>/<volume>/t.txt' OVERWRITE")

Questions

Is SQL connector PUT from inside serverless compute supported? The volume docs list PUT under external tools, and the serverless limitations page doesn't mention it either way.
If it is supported, is the private DNS answer for eastus.storage.azuredatabricks.net on serverless a known issue (East US / az-eastus-c3)?
Has anyone seen this in other regions?

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

SumeshKashyap — Mon, 26 Oct 2026 11:52:24 GMT

Hi @sudopradip

Short version: the SQL connector's PUT/GET is a staging path meant for external clients. From inside serverless, the storage-gateway hostname resolves through the serverless plane's private DNS to an endpoint that doesn't serve that traffic, so you can't fix it from your code. Inside Databricks, write to the volume through FUSE or the Files API (both already work for you), and open a support ticket for the DNS behaviour.

On your questions:

1. Supported from inside serverless? I couldn't find any doc that says it is. The volumes docs list PUT/GET INTO under connecting from external tools and drivers. For code running on Databricks compute, the documented options are the /Volumes FUSE path and the Files API (WorkspaceClient().files.upload). Your classic-compute success fits this: classic resolves the gateway through normal Azure DNS, while serverless resolves it inside the Databricks-managed network.

2. Private IP for eastus.storage.azuredatabricks.net: you've shown the gateway is reachable on its public IP and only the in-plane DNS answer is wrong, and it happens in a fresh default workspace with no NCC. That puts it on the platform side. Raise it with Azure Databricks support and include the workspace IDs, region/sn=az-eastus-c3, the 192.168.200.28 vs 52.234.168.242 resolution, and your minimal repro. Don't hardcode the public IP as a workaround: those IPs can change, and it bypasses whatever routing serverless intends.

3. Other regions: I haven't tested this outside East US.

Workarounds for the dlt load in the meantime:

- Run the dlt pipeline from classic (job) compute, where direct load works as you've seen.

- Or configure dlt staging to an ADLS Gen2 container that's a UC external location (filesystem staging with an az:// bucket_url plus staging credentials). The databricks destination then loads with COPY INTO from that location and never calls the PUT gateway.

- If you control the load code, write files with shutil.copy to /Volumes/... or files.upload(), then run COPY INTO / read_files from the volume path.

Docs: https://learn.microsoft.com/azure/databricks/volumes/volume-files and https://dlthub.com/docs/dlt-ecosystem/destinations/databricks

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

SumeshKashyap — Mon, 26 Oct 2026 14:06:21 GMT

hello @sudopradip , if the response was helpful, please accept the reply as answer so that it is beneficial for other community members

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

sudopradip — Mon, 26 Oct 2026 14:35:53 GMT

Update: narrowed scope. It fails when the catalog is on Databricks-managed storage (storage Databricks owns, outside our Azure subscription, which a serverless workspace uses for a catalog created with no location). In a hybrid workspace, the catalog Databricks creates automatically sits on the workspace's own storage account in our Azure subscription; with that catalog, the same stock dlt load from serverless compute works.

thanks @SumeshKashyap
that make sense
i think i will keep issue open until it's officially documented or confirmed

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

data_pulse — Mon, 26 Oct 2026 15:45:27 GMT

I tested this independently and was able to reproduce a very similar failure pattern on AWS us-east-2, using a Databricks serverless.

Here are the results against the same UC Volume:

Upload method Result

Direct /Volumes/.. write	PASS (byte readback verified)
Databricks SDK Files API (0.40.0)	PASS (byte readback verified)
SQL Connector 4.8.0 PUT INTO	FAIL (twice)
SQL Connector 4.4.0 PUT INTO	FAIL (twice)
SQL Connector 4.8.0 from local Windows	PASS (twice, byte readback verified)

The interesting part is the gateway connectivity:

Inside the serverless notebook, us-east-2.storage.cloud.databricks.com resolved to 192.168.200.28, with TCP port 443 returning [Errno 111] Connection refused.
Warehouse authentication and simple SELECT succeeded, so the failure occurred later, during the connector's storage-gateway upload.
From local Windows, the same gateway resolved to public IP addresses, TCP 443 succeeded, and uploads to the same warehouse and Volume worked

This confirms that similar symptoms beyond Azure East US, including on AWS us-east-2.

It also suggests the behaviour is not limited to SQL Connector 4.8.0.

The results point toward a storage gateway connectivity issue specific to the serverless execution environment, rather than general UC Volume permissions or SQL warehouse connectivity.

Hopefully this cross-cloud reproduction helps narrow down the investigation.

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

sudopradip — Thu, 29 Oct 2026 06:13:01 GMT

Thanks @data_pulse 🙌 This helps a lot! We can now rule out my Databricks setup as the cause; it's a real but.

The fact that it works from Windows points to how the core service handles Databricks-managed storage access from notebooks, narrow down the issue.

Re: Serverless notebook: UC volume PUT via SQL connector fails *.storage.azuredatabricks.net resolve

data_pulse — Thu, 29 Oct 2026 08:41:55 GMT

Welcome @sudopradip . Glad it helped to narrow down further.