Workspace folder is visible but .py file cannot be read on job cluster (DBR 18)

[Annie420]

Hi everyone,
We are running into a strange issue when running notebooks on Databricks job clusters using DBR 18. It looks like the Workspace folder is mounted, but the .py file inside cannot be read immediately. I wanted to check if anyone else has experienced this or knows a recommended workaround.

The file is located here:

/Workspace/Shared/***/***/WidgetUtil.py. Inside the notebook we try to check if the file exists and then read it.

What we observe:

When the job cluster starts:

• The Workspace folder appears to exist
• os.path.exists("/Workspace/...") returns True
• But trying to open the file fails. So the system thinks the folder is mounted, but the file cannot be read yet.

The output looks something like:

Notebook cwd: /Workspace/Shared/***/***
WidgetUtil exists: True
Workspace exists: True
Waiting for workspace file:
/Workspace/Shared/***/***/WidgetUtil.py
Exception: Workspace file not available after 120 seconds

Strange behavior:

If we run the same job later (sometimes an hour later):

• The file becomes readable immediately
• No code changes required

So it looks like the folder is mounted but the file contents are not available yet.

Additional notes:

• This issue happens mainly on job clusters
• On interactive clusters, the file is available immediately
• We are using Databricks Runtime 18

Questions:

• Is this expected behavior with Workspace FUSE mounts on job clusters on 18 DBR?
• Is there a recommended way to reliably access .py files from /Workspace in jobs?

We would really appreciate any guidance or best practices.

[pradeep_singh]

Can you try this method for reading workspace files .

https://docs.databricks.com/aws/en/files/workspace-interact

If you can , use git folders instead of workspace files fi above method doesn't work for some reason.

[pradeep_singh · Accepted solution]

The error might be because of delay in workspace files being accessible . 

The /Workspace mount point appears quickly, but the FUSE daemon may still be initializing auth, metadata, and connections to the workspace storage account.

FUSE = Filesystem in Userspace: a Linux mechanism where a user‑space daemon implements a filesystem that the kernel exposes as a normal mount point.On Databricks, paths like /Workspace/... (workspace files) and /Volumes/... (Unity Catalog volumes) are exposed via a FUSE layer .

[stbjelcevic · Accepted solution]

+1 to @pradeep_singh 

The Workspace FUSE (WSFS) daemons use ports 1015, 1017, and 1021 for communication between the driver and the executor. NFS tooling (hardcoded in glibc) can race with these ports during cluster startup, causing FUSE daemons to fail to bind. This explains the intermittent nature, sometimes the port race doesn't happen and it works fine.

On interactive clusters, the driver accesses /Workspace via a local FUSE mount. On multi-node job clusters, executors must RPC to the driver over those ports (a fundamentally different code path).

Check your VPC security group rules to ensure all TCP ports are open between nodes in the same security group (if you are using a managed VPC).