szymon_dybczak
Esteemed Contributor III

Hi @MikeGo ,

Yes, you are correct. foreachBatch operates on a single batch at time, so you can safely use drop duplicates without worrying about state management.

View solution in original post