- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2025 09:00 PM
Hi Big Roux,
Following up on my previous response, I think it might help if I describe the situation more clearly.
The pipeline starts with read_stream from a Delta table and create a view without applying watermarking.
The next stage involves using create_streaming_table and the apply_flow. I'm trying to read from different source and create separate views, but I need to aggregate the result after doing some transformation on these sources.
Delta_tables -> Views -> Streaming_table(append_flow)
In this case, do I need to apply the water marking in the first step when I try to read stream from my actual source(the Delta table)?
I hope this could give you a clearer picture on how my current pipeline works.
Thank you.
Regards,
Dejian