topic Re: ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE in Data Engineering

ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

yatharthmahesh — Tue, 02 Aug 2022 10:21:49 GMT

I have a delta table already created, now I want to enable the change data feed. I read that I have to set delta.enableChangeDataFeed property to true. But however, this cannot be done using the Scala API. I tried using this but it didn't work. I am using the OS version (delta) for performing all these operations.

I really appreciate any help you can provide.

val ddl_query = """CREATE TABLE if not exists delta_training.delta_table
                   USING DELTA
                   LOCATION '/Users/data-partner-merge/src/main/resources/delta/dpm/base'
                   """
 //creation of table
 spark.sql(ddl_query)
 
 spark.sql(" ALTER TABLE delta_training.onaudience_dpm SET TBLPROPERTIES ('delta.enableChangeDataFeed' = true )").show()

EDIT 1: As mentioned by a user, I tried using this as well, but still no changes come into action.

spark.sql(" ALTER TABLE delta_training.onaudience_dpm SET TBLPROPERTIES (delta.enableChangeDataFeed = true)")

Re: ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

Hubert-Dudek — Tue, 02 Aug 2022 13:27:05 GMT

'delta.enableChangeDataFeed' have to be without quotes.

 spark.sql("ALTER TABLE delta_training.onaudience_dpm SET TBLPROPERTIES (delta.enableChangeDataFeed = true)").show()

Re: ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

yatharthmahesh — Tue, 02 Aug 2022 15:44:43 GMT

I tried but that doesn't work

Re: ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

jose_gonzalez — Tue, 30 Aug 2022 17:15:41 GMT

Can you try in plain SQL instead of spark.sql() API? According to the docs, this is the command to do it :

ALTER TABLE myDeltaTable SET TBLPROPERTIES (delta.enableChangeDataFeed = true)

docs https://docs.databricks.com/delta/delta-change-data-feed.html#enable-change-data-feed