Issue with MERGE INTO for first batch
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-26-2023 06:33 PM
I have source data with multiple rows and columns, 1 of column is city.
I want to get unique city into other table by stream data from source table. So I trying to use merge into and foreachBatch with my merge function. My merge condition is : On source.city = target.city. But then I realized that, in first batch, I will always get duplicated data becauce there is no data on target and duplicated city on first batch of source.
How can I resolve this ? thank you for supporting