List and iterate over files in Databricks workspace

semsim
Contributor

‎08-07-2024 01:05 PM

Hi DE Community,

I need to be able to list/iterate over a set of files in a specific directory within the Databricks workspace. For example:

"/Workspace/SharedFiles/path/to/file_1"

...

"/Workspace/SharedFiles/path/to/file_n"

Thanks for your direction and help

0 Kudos

szymon_dybczak
Esteemed Contributor III

‎08-08-2024 12:22 AM

Hi @semsim ,

You can use File system utility (dbutils.fs)

Databricks Utilities (dbutils) reference | Databricks on AWS
Work with files on Databricks | Databricks on AWS

dbutils.fs.ls("file:/Workspace/Users/<user-folder>/")