I have a delta live table where I am reading cdc data and merging this data in silver using apply changes. In silver can I find out what all data has changed since the last run similar to change data feed table_changes?
Hi DatabricksCould we please regarding information about upcoming sessions and recorded videos of past sessions .This would be v helpful for users like meKind RegardsPriya
I think my main problem is that i havent been able to enable change data feed on the silver layer since its a view..dlt.create_target_table(f"silver_{schemaName}_{tableName}",table_properties = {'delta.enableChangeDataFeed': 'true'})I can see the cha...
I also have a requirment where i write to a live table (materialized view) and have cdf enabled i want to see the changes but here to i see overwrites happening after dlt pipeline runs
thanks for your reply @Werner Stinckens i have enabled change data feed but the issue is every time i run the dlt pipeline an overwrite happens so when i do table_changes between versions i see all the changes as inserts?
@Suteja Kanuri From my understanding When you create a materialized view, its contents reflect the state of the underlying database table or tables at that time. The data in the materialized view remains unchanged, even when applications make change...
Isnt this in private prview currently https://www.databricks.com/blog/2022/07/22/power-to-the-sql-people-introducing-python-udfs-in-databricks-sql.html
