DLT streaming table and LEFT JOIN
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2022 09:16 AM
I'm trying to build gold level streaming live table based on two streaming silver live tables with left join.
This attempt fails with the next error:
"Append mode error: Stream-stream LeftOuter join between two streaming DataFrame/Datasets is not supported without a watermark in the join keys, or a watermark on the nullable side and an appropriate range condition".
My question is how I can implement the watermark with SQL interface? Where can I find examples?
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-12-2022 01:02 AM
Thanks Fatma,
I do understand the need for watermarks, but I'm just wondering if this supported by SQL syntax?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-24-2023 11:45 AM
I'm wrestling with similar questions. This article makes it pretty clear that you'll need to use scala/python/java/r: streaming writes
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-24-2023 02:18 PM