I have an S3 bucket that has continuous data being written into it. My script reads these files, parses them and then appends into a delta table. 

The data backs to 2022 with millions of files which are stored using partitions based on year/month/dayOfMonth/hourOfDay.

Up until now, I have been using previous day as a filter to read the data and process it. However, now I want to switch to incremental batch streaming using directory listing autoloader. How do I switch to it without having the need to parse the entire S3 to create the initial checkpoint?