Unable to see a folder in DBFS

[Shivani_Komma99]

Hi Team,

We have a few scripts stored in a folder on a DBFS path. Recently, we've noticed that when we navigate to this path manually, the folder appears to be empty, and we are unable to see the scripts.

However, the jobs that reference and access these scripts are still running successfully. There have been no updates or deletions made to these files from our side.

Could you please help us understand why this is happening? This is important because we need to upload another file to the same DBFS folder for our upcoming jobs to run successfully.

[Ale_Armillotta]

Hi  @Shivani_Komma99 

What you're describing is consistent with a known behavior in Unity Catalog-enabled workspaces.

The most likely explanation is related to cluster access mode:

• Compute resources configured with dedicated access mode have full access to DBFS, including all files in the DBFS root and mounted data. This is probably why your jobs are still running successfully.

• In standard access mode, to interact with files directly using DBFS, users must have ANY FILE permissions explicitly granted. Without it, the folder may appear empty in the UI even though the files are physically there.

Also worth noting: both DBFS root and DBFS mounts are deprecated and not recommended by Databricks. New accounts are provisioned without access to these features.

A few questions to better understand your setup:

1. What cluster access mode are your jobs running on — dedicated (single-user) or standard (shared)?

2. How are you navigating to the DBFS path — via the UI file browser, dbutils.fs.ls(), or the CLI?

3. Is Unity Catalog enabled in your workspace?

This will help us point you to the right solution for uploading the new file safely. Thanks!

[balajij8]

The underlying storage & files remains active with no issues and hence the production jobs continue to execute without failures. Cluster runtimes retain programmatic access through the legacy world (dbfs) even though the assets are not visible from the graphical user interface. You can ask the workspace admin to check Settings - Advanced - DBFS File Browser toggle as they may have disabled it (you cannot see in visual interface if disabled) the DBFS File Browser toggle in workspace settings for robust governance. More details here

You can utilize programmatic ingestions into the path. You can immediately migrate all scripts and file dependencies into Unity Catalog Volumes as its robust. You can use a governed 3 level namespace (catalog/schema/volume/) in the code and replace the old dbfs legacy paths with robust volumes.

[Shivani_Komma99]

Hi Balaji,

We can see the DBFS browser, but a particular folder is not visible in it.

Interestingly, the same folder is visible in our Production environment but not in the Development environment. However, the jobs in both environments are able to access the files in this folder and run successfully.

We are also able to download the files from that folder, and we believe uploads may also work. We would like to understand the underlying cause of why the folder is not accessible or visible through the UI, despite being accessible programmatically.

Could you please help us identify the reason for this behavior?

Thank you.

[Ashwin_DSA · Accepted solution]

Hi @Shivani_Komma99,

Thanks for flagging this. Based on the behaviour you described, this appears to be consistent with a DBFS browser UI issue/regression rather than a problem with the underlying files, especially since the files are still accessible from jobs/programmatically.

Please DM me your workspace ID, cloud/region, the exact DBFS path, and a screenshot or short recording of the UI flow so I can confirm whether you’re hitting the same issue and advise on the best workaround/next steps.

In the meantime, if you need to unblock an upload urgently, using CLI or another programmatic method is likely the safest path.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.