Variant should work in this scenario too. There's also been some performance improvements with variant so much more of the metadata has stats for efficient processing. 

You also don't have to go file by file. You can use things like autoloader that will checkpoint all the reads, or if you want you can use "*" in a location to denote everything in that path.