APPLY_CHANGES late arriving data

[Gilg]

Hi Team,

I have a DLT pipeline that uses APPLY_CHANGES to our Silver tables. 

I am using Id as keys and timestamp to know the sequence of the incoming data. 

Question: How does APPLY_CHANGES handles late arriving data?

i.e., for silver_table_1, the data comes for Id 1 at 08/10/2023 at 10:30pm. This record got inserted to silver_table_1. Next day, same Id 1 data arrives but the timestamp is 07/10/2023 at 08:00pm. As you can see the timestamp is different and is less than from the inserted record. 

Cheers,

G

[Gilg]

Hi Kaniz,

Will this work in SCD Type1 table? As I understand the __START_AT and __END_AT columns are only available in SCD Type 2 APPLY_CHANGES. When I do a quick select of the table using SQL Warehouse I do not see these columns in a Type 1 table.

Cheers,

G