Enabling enableChangeDataFeed on Streaming Table created in DLT

[Vsleg]

Hello, 

Can I enable Change Data Feed on Streaming Tables? How should I do this? I couldn't find this in the existing documentation https://learn.microsoft.com/en-us/azure/databricks/delta/delta-change-data-feed .

[Kaniz_Fatma]

Hi @Vsleg, Enabling the Change Data Feed feature for streaming tables in Azure Databricks allows you to track row-level changes between versions of a Delta table. When enabled, the runtime records change events for all data written into the table, including metadata indicating whether a row was inserted, deleted, or updated. You can then read these change events in batch queries using Spark SQL, Apache Spark DataFrames, and Structured Streaming.

Here’s how you can enable the Change Data Feed:

1. New Table:

   • When creating a new table, set the table property delta.enableChangeDataFeed = true in the CREATE TABLE command. For example:

     CREATE TABLE student (id INT, name STRING, age INT)
     TBLPROPERTIES (delta.enableChangeDataFeed = true);
     

2. Existing Table:

   • If you already have a Delta table, you can enable the Change Data Feed by setting the table property using the ALTER TABLE command. For example:

     ALTER TABLE myDeltaTable SET TBLPROPERTIES (delta.enableChangeDataFeed = true);
     

3. All New Tables:

   • To enable the Change Data Feed for all new tables, set the Spark configuration property spark.databricks.delta.properties.defaults.enableChangeDataFeed = true.

Remember that only changes made after enabling the Change Data Feed are recorded; past changes to a table are not captured. Additionally, the Change Data Feed works alongside table history to provide comprehensive change information. Cloning a Delta table creates a separate history, so the Change Data Feed on cloned tables won’t mat...¹.

[Vsleg]

Hello Kaniz,

I already tried this ALTER TABLE but it doesn’t work for Streaming Table creating inside a DLT pipeline.

[Vsleg]

I got this error and tried CREATE OR REFRESH as well but that didn't work either..

[Vsleg]

@Kaniz_Fatma ?