raphaelblg
Databricks Employee
Databricks Employee

Hello @Devsql 

Autoloader initially lists files using one of the File Detection Modes. For each batch of files discovered, a checkpoint is created. If you wish to examine the state of your checkpoint, you can use the cloud_files_state SQL function, which displays all files discovered by Autoloader.

Autoloader uses Checkpointing to maintain state, maintaining exactly-once processing guarantees throughout your spark structured streaming query.

I hope my answer is helpful to you, I've attempted to provide a comprehensive overview of Databricks Autoloader's features.

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks