topic Re: Multiple streaming sources to the single delta live table in Data Engineering

Multiple streaming sources to the single delta live table

Sinthiya — Fri, 11 Aug 2023 08:07:40 GMT

In our case, we have multiple sources writing to the same target table. A target table can be populated from multiple source tables, each contributing a set of fields.

How to add/update columns in a target table from multiple sources.

In a delta live table, how to push/pop objects from an array column. Arrays from multiple sources as well.

SQL STATEMENT :

CREATE OR REFRESH STREAMING TABLE member_gold;

APPLY CHANGES INTO

live.member_gold

FROM

stream(LIVE.silver_contacts_fromcontactsfile)

KEYS

(clientMemberId)

SEQUENCE BY

clientMemberId

STORED AS

SCD TYPE 1;

APPLY CHANGES INTO

live.member_gold

FROM

stream(LIVE.silver_demographic)

KEYS

(clientMemberId)

SEQUENCE BY

clientMemberId

STORED AS

SCD TYPE 1;

Re: Multiple streaming sources to the single delta live table

SaiKiranGajjala — Fri, 11 Aug 2023 08:49:52 GMT

Following.