dazfuller
Databricks Partner

So one way of doing this is to simple iterate over the directories at that level and create a table for each. Assuming that the schema is the same in all files you could simple do something like

subjects = dbutils.fs.ls("abfss://landing@account.dfs.core.windows.net/sc/raw")
 
for subject in subject:
    table_name = subject.name.strip("/")
    spark.sql(f"CREATE TABLE IF NOT EXISTS `{table_name}` USING PARQUET LOCATION '{subject.path}'")

View solution in original post