Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-12-2026 07:39 AM
Hi everyone,
We are running into a strange issue when running notebooks on Databricks job clusters using DBR 18. It looks like the Workspace folder is mounted, but the .py file inside cannot be read immediately. I wanted to check if anyone else has experienced this or knows a recommended workaround.
The file is located here:
/Workspace/Shared/***/***/WidgetUtil.py. Inside the notebook we try to check if the file exists and then read it.
What we observe:
When the job cluster starts:
- The Workspace folder appears to exist
- os.path.exists("/Workspace/...") returns True
- But trying to open the file fails. So the system thinks the folder is mounted, but the file cannot be read yet.
The output looks something like:
Notebook cwd: /Workspace/Shared/***/***
WidgetUtil exists: True
Workspace exists: True
Waiting for workspace file:
/Workspace/Shared/***/***/WidgetUtil.py
Exception: Workspace file not available after 120 seconds
WidgetUtil exists: True
Workspace exists: True
Waiting for workspace file:
/Workspace/Shared/***/***/WidgetUtil.py
Exception: Workspace file not available after 120 seconds
Strange behavior:
If we run the same job later (sometimes an hour later):
- The file becomes readable immediately
- No code changes required
So it looks like the folder is mounted but the file contents are not available yet.
Additional notes:
- This issue happens mainly on job clusters
- On interactive clusters, the file is available immediately
- We are using Databricks Runtime 18
Questions:
- Is this expected behavior with Workspace FUSE mounts on job clusters on 18 DBR?
- Is there a recommended way to reliably access .py files from /Workspace in jobs?
We would really appreciate any guidance or best practices.
.png){kind=link}
.png){kind=link}