Incremental load from a sql table

[AKon]

Hi, Is there a way to read a sql table using jdbc as a streaming source? If not what is the best approach to read incremental data from a sql table and implement SCD Type 1 while loading into silver and gold layer. Can this be implemented using dlt pipeline?

[MariuszK]

hi,

You can use ADF to extract data incrementally into the bronze layer, and use DLT to load this data into silver with SCD1 or SCD2.

You will need to store somewhere information about last extracted ID and read it in ADF pipeline to get only new records.