Manjunath
Databricks Employee
Databricks Employee

Hi @Leszek​ 

For your case ignoreChanges option will work, but you need to handle duplicates from your streaming app while writing to sink. If your sink is Delta then you can go with Delta Streaming Merge.

https://docs.databricks.com/_static/notebooks/merge-in-streaming.html

View solution in original post