We found that when using os.listdir() in Databricks notebooks to list files stored in the Workspace (i.e., alongside the notebook, not in DBFS), file extensions were missing in Databricks Runtime 14.3 LTS and 15.4 LTS, but appeared correctly in 16.4 LTS.

For example os.listdir('.') would return the following code for the different runtimes:

>>> os.listdir('.') # DBR 14.3LTS, DBR 15.4LTS
['notebookA', 'fileB', 'notebookc']

>>> os.listdir('.') # DBR 16.4LTS, Serverless compute
['notebookA.sql', 'fileB.py', 'notebookc.ipynb']

 On the code level we can handle this, but I'd like to understand and/or track down this breaking change between the different runtimes.

Can you help me with the reason?