Hello, I am using databricks autoloader with managedfileevents turned on and include existing files.

I want to understand if there is a way of increasing the speed of the initial listing of the files for autoloader.
I thought that the idea behind the managed file events mode was that essentially Databricks stores a list of all the file events that have come in and when you create an autoloader instance to read from a path under the container that has the fileevents enabled. Autoloader will make a custom queue to for that instance of autoloader and replay the events.

The issue I have is that when running large tests the speed of the include existing files is very slow and needs to be redone if for example I created another instance of autoloader to write to another table. Is there a way of passing in the list of existing files or increasing the number of processes that are listing files e.g. at each folder level adding another process to walk that directory.