topic List and iterate over files in Databricks workspace in Data Engineering

List and iterate over files in Databricks workspace

semsim — Wed, 07 Aug 2024 20:05:47 GMT

Hi DE Community,

I need to be able to list/iterate over a set of files in a specific directory within the Databricks workspace. For example:

"/Workspace/SharedFiles/path/to/file_1"

...

"/Workspace/SharedFiles/path/to/file_n"

Thanks for your direction and help

szymon_dybczak — Thu, 08 Aug 2024 07:22:42 GMT

You can use File system utility (dbutils.fs)

dbutils.fs.ls("file:/Workspace/Users/<user-folder>/")