Enable CDC in Lakeflow Connect Tables

[Nmtc9to5]

Hello everyone, I'm implementing a project that requires AutoCDC workflows using Lakeflow.

The architecture is as follows: Data is ingested from a database using Lakeflow Connect, and then a declarative pipeline performs some transformations on this data to load it into a streaming table.

The declarative pipeline needs to capture the changes made to the streaming table generated by the Lakeflow Connect pipeline. So, how can I enable the `delta.enable.ChangeDataFeed` property on the table generated by Lakeflow Connect? Is there a way to do this? Or do these tables already contain the necessary CDC metadata by default?

Thanks in advance for your answers.

[stbjelcevic]

Hi @Nmtc9to5 ,

You likely don't need to manually enable delta.enableChangeDataFeed on Lakeflow Connect tables for most use cases, as they typically contain the CDC metadata by default. (docs)

You can check if it is enabled on the table with SHOW TBLPROPERTIES catalog.schema.your_table;

Use STREAM() in your downstream declarative pipeline to incrementally read from the Lakeflow Connect table.