Hello!
I have a (non-stateful) daily structured streaming ingestion job from json S3 file storage into a Delta table using Spark readStream. I would like to switch to Auto Loader to reduce file discovery time. My questions are:
- Do I need to create a new streaming checkpoint? (I am guessing the answer to this question is yes)
- Can I keep appending to the same Delta table (and table location), or do I need to create a new table as well?
Thank you!
Cosimo.
