Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs

nopal1 — Tue, 05 Aug 2025 11:26:34 GMT

We found that when using os.listdir() in Databricks notebooks to list files stored in the Workspace (i.e., alongside the notebook, not in DBFS), file extensions were missing in Databricks Runtime 14.3 LTS and 15.4 LTS, but appeared correctly in 16.4 LTS.

For example os.listdir('.') would return the following code for the different runtimes:

>>> os.listdir('.') # DBR 14.3LTS, DBR 15.4LTS ['notebookA', 'fileB', 'notebookc'] >>> os.listdir('.') # DBR 16.4LTS, Serverless compute ['notebookA.sql', 'fileB.py', 'notebookc.ipynb']

On the code level we can handle this, but I'd like to understand and/or track down this breaking change between the different runtimes.

Can you help me with the reason?

Re: Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs

cgrant — Thu, 07 Aug 2025 22:40:35 GMT

This is expected and changed in DBR16.2:

In Databricks Runtime 16.2 and above, notebooks are supported as workspace files.

Re: Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs

nopal1 — Sat, 09 Aug 2025 04:37:51 GMT

Thank you so much!

topic Re: Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs in Data Engineering

Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs

Re: Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs

Re: Python os.listdir() behavior difference between 15.4LTS and 16.4LTS DBRs