Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @der ,

I have checked internally. No specific changes to workspace files access were introduced as part of DBR 18.1. The error seems to indicate that the mount/backend is unhealthy or blocked (network/NSG/cluster config), not that your Python code is wrong.

Can you try the below...

import os, getpass, subprocess, textwrap

user = getpass.getuser()
path = f"/Workspace/Users/{user}"

# Does listing the directory itself fail?
print("Listing:", path)
print(subprocess.run(["ls", "-l", path], text=True, capture_output=True))

If "ls" itself throws the same error, the /Workspace mount is broken on this cluster.

You can also spin up a new cluster with minimal config (no custom VNet/NSG changes, no init scripts, no extra drivers) and run the same open("/Workspace/...") there. If it works on the clean cluster but not on the original one, it’s a cluster/network configuration issue.

A robust workaround is to store configs on DBFS instead of raw /Workspace/Users/.. However, if you must use the workspace files... the officially supported way is through the Workspace Files API / WorkspaceClient().files, rather than relying on FUSE paths. Use the SDK to download the file to local disk, then json.load it.

If the above doesn't resolve your issue, this is a good candidate for a Databricks support case. As you already have it, attach the trace ID, cluster ID, workspace URL, and confirm that any customer‑managed NSGs allow egress to workspace APIs (including /api/2.0/workspace-files/*).

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***