Sidhant07
Databricks Employee
Databricks Employee

Yes, using autoloader with file notification mode can be useful here.

Also you can use the IF EXISTS clause to check if the files exist before attempting to create or refresh the materialized view. This will prevent the SQL block from running if there are no new files.