Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @der,

Can you run the same notebook on a test cluster in the same workspace with DBR 18.0 (or 17.3 LTS) with same access mode, DBFS still disabled? If the issue disappears there and only occurs on 18.1, that’s strong evidence for an 18.1‑specific issue.

If you hit the error on 18.1.. just capture the trace ID from the stack trace and immediately run the below in a Python cell.

import os, getpass, subprocess

user = getpass.getuser()
base = f"/Workspace/Users/{user}"

print("ls base:")
print(subprocess.run(["ls", "-l", base], text=True, capture_output=True))

print("ls file:")
print(subprocess.run(["ls", "-l", "/Workspace/Users/<USER>/config.json"], text=True, capture_output=True))

In a %sh cell, check for FUSE / workspace-files errors:

dmesg | tail -n 50 || true

You can then open a support ticket with the Workspace URL and region, Cluster ID, DBR: 18.1, access mode, DBFS disabled, the full Python stack trace with trace ID(s), Output of the ls / dmesg checks above on a failing run and also provide confirmation that the same code on 18.0 or 17.3 LTS does not show Errno 5.

That gives support/engineering exactly what they need to confirm a regression in 18.1’s WSFS/FUSE behaviour.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***