cdc type 2 - not closing records

MariuszJ
New Contributor II

Hi,
i would not mind small advice, i do have dlt cdc typ2, the definition

 

dlt.create_streaming_table('`my_table_dlt_cdc`')
dlt.apply_changes(
  target = 'my_table_dlt_cdc',
  source = 'source',
  keys = ['id'],
  sequence_by = col('snapshot_date'),
  # ignore_null_updates = True,
  except_column_list = ['snapshot_date'],
  stored_as_scd_type = "2"
)

 

the table it self is very simple

 

id| val    | snapshot_date
1 | 'abc' | 2023-01-01

 

The '1' is stopped to exist in source table after march, but cdc never closed the row. Data for whole 2023 has been loaded (up to today).

When querying __apply_changes_storage_my_table_dlt_cdc i do see last entry in march but nothing after, row was not closed.

 

 

id,value,__recordStartAt,__START_AT,__END_AT,__rowIsHidden
1,'ABC',2023-03-17,2019-07-05,null,true
1,'ABC',2023-03-24,2019-07-05,null,true
1,'ABC',2023-03-31,2019-07-05,null,false

 

I do have a feeling that I'm missing something obvious... But what.

MariuszJ
New Contributor II

For anyone who sees this post in the future. I was missing one argument

apply_as_deletes