Dejian
New Contributor II

Hi Big Roux,

Following up on my previous response, I think it might help if I describe the situation more clearly.

The pipeline starts with read_stream from a Delta table and create a view without applying watermarking.
The next stage involves using create_streaming_table and the apply_flow. I'm trying to read from different source and create separate views, but I need to aggregate the result after doing some transformation on these sources.

Delta_tables -> Views -> Streaming_table(append_flow)

In this case, do I need to apply the water marking in the first step when I try to read stream from my actual source(the Delta table)?

I hope this could give you a clearer picture on how my current pipeline works.

Thank you.

Regards,

Dejian