Lakshay
Databricks Employee
Databricks Employee

@therealchainman 

The backFillInterval option is provided to make sure eventually all the files are inserted. When you create a new stream, some files might be missed that are not ingested. BackFill is an asynchronous process which is trigerred based on the interval defined by backFillInterval option. This checks for all the files that have been missed and ingests those files