I have source data with multiple rows and columns, 1 of column is city.
I want to get unique city into other table by stream data from source table. So I trying to use merge into and foreachBatch with my merge function. My merge condition is : On source.city = target.city. But then I realized that, in first batch, I will always get duplicated data becauce there is no data on target and duplicated city on first batch of source.
How can I resolve this ? thank you for supporting
