@DylanStout
Since dbutils.fs.ls("dbfs:/mnt") shows your files but the Databricks UI folder view keeps loading indefinitely, the issue likely stems from:
- UI rendering issue
- Large directory size or high file count
- Latency in metadata retrieval
- Permissions inconsistency
Recommended Steps to Resolve the Issue
1. Check Folder Size / File Count
The UI may struggle to render folders with thousands of files or deeply nested structures.
Solution: Use dbutils.fs.ls() in combination with len() to check file counts:
len(dbutils.fs.ls("dbfs:/mnt/<folder>"))
If the count is excessively high, consider partitioning or reorganizing files to improve UI responsiveness.
2. Verify Folder Permissions
Even if dbutils.fs.ls() works, the UI may fail due to missing read/list permissions in Unity Catalog or Storage ACLs.
Solution: Run the following to inspect permissions:
SHOW GRANTS ON STORAGE LOCATION '/mnt/<mount-point>';
If permissions are missing, add them:
GRANT READ ON STORAGE LOCATION '/mnt/<mount-point>' TO `<user/role>`;
3. Check for Mount Point Misconfiguration
An unstable or partially broken mount could cause this issue.
# Unmount (if already mounted)
dbutils.fs.unmount("/mnt/<mount-point>")
# Remount
dbutils.fs.mount(
source = "wasbs://<container>@<storage-account>.blob.core.windows.net/",
mount_point = "/mnt/<mount-point>",
extra_configs = {"<conf-key>":dbutils.secrets.get(scope="<scope>", key="<key>")}
)
4. Cluster Driver Overload / Memory Issue
UI delays can occur if the clusterโs driver node is overloaded.
Solution:
Restart the cluster.
If possible, increase the driver size (especially for metadata-heavy operations).
5. Force Refresh Metadata
Sometimes stale metadata can cause UI issues.
Solution: Run this command to force metadata sync:
ALTER TABLE <catalog>.<schema>.<table_name> REFRESH;