topic DLT- apply_changes() SCD2 - Rename_columns : __START_AT & __END_AT in Data Engineering

DLT- apply_changes() SCD2 - Rename_columns : __START_AT & __END_AT

VenkateswarluAd — Mon, 15 Jul 2024 17:21:29 GMT

Renaming the column names of the __START_AT and __END_AT columns created when using the dlt.apply_changes() method for performing SCD2 type updates.

Re: DLT- apply_changes() SCD2 - Rename_columns : __START_AT & __END_AT

szymon_dybczak — Mon, 15 Jul 2024 17:43:11 GMT

Hi @VenkateswarluAd ,

I don't think it is possible to change those names. They are used internally by dlt framework and if you change them something definitely gonna break.

Re: DLT- apply_changes() SCD2 - Rename_columns : __START_AT & __END_AT

Ravivarma — Mon, 15 Jul 2024 18:16:26 GMT

Hello @VenkateswarluAd ,
Greetings of the day!

The columns __START_AT and __END_AT are used to track the validity period of each record for SCD Type 2 updates. Please be aware that renaming these columns could disrupt the functionality of the SCD Type 2 updates.

Regards,

Ravi