Delta Live Tables - how do you merge multiple streaming datasets into a single one?

[Ian]

I need to create a dataset that is dependent on multiple streaming datasets. However, when I attempt to create the new single stream I am getting an error.

Append output mode not supported when there are streaming aggregations on streaming DataFrames/DataSets without watermark;

Example:

create or refresh streaming live table silver_customer;

create temporary streaming live view customer_updates

as

with listOfCustomers as

(

select CustomerID

from

 stream(live.raw_Customer)

union

select CustomerID

from

 stream(live.raw_CustomerAddress)

)

select CustomerID

from listOfCustomers;

apply changes into 

 live.silver_customer

from

 stream(live.customer_updates)

keys(CustomerID)

sequence by CustomerID

stored as scd type 1;

[Kaniz]

Hi @Ian Corera​, This S.0 thread has a similar discussion. Please let us know if that helps.

[jose_gonzalez]

Hi @Ian Corera​,

Just a friendly follow-up. Do you still need help? please let us know if you do.

[Ian]

Hi Kaniz/Jose, I was able to resolve the issue. I used 'union all' to avoid aggregation on the stream and have it continue to write to the table in append mode.

This issue can be closed.