Consume updated data from the Materialized view and send it as append to a streaming table
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-24-2024 08:25 AM - edited 03-24-2024 08:28 AM
Hello everyone! I'm using DLT and I'm pretty new to them. I'm trying to take the updates from a materialized view and send them to a streaming table as an append.
For example, if I have a MV of 400 records, I want an append to be made to the streaming table and when the view is updated again, an append to the streaming table is made again, always having a total of 400 in the MV and the streaming rate increases, in this case it would reach 800 for the two MV updates.
I'm currently trying to do it and I have this error:
Flow 'streaming_table_test' has FAILED fatally. An error occurred because we detected an update or delete to one or more rows in the source table. Streaming tables may only use append-only streaming sources. If you expect to delete or update rows to the source table in the future, please convert table streaming_table_test to a live table instead of a streaming live table. To resolve this issue, perform a Full Refresh to table streaming_table_test. A Full Refresh will attempt to clear all data from table kafka_events and then load all data from the streaming source. The non-append change can be found at version 5192.