- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-08-2023 06:16 PM
@Ram LH :
The error message suggests that there were updates or deletions made to the source table, which is not supported in streaming tables. Streaming tables in Databricks are meant to be append-only and any updates or deletions to the source table can result in data inconsistencies in the streaming table.
To resolve this issue, you can perform a full refresh of the target table instead of an incremental load. Also, you should convert the target table to a live table instead of a streaming live table if you expect updates or deletions to the source table in the future.
You can also consider implementing a change data capture (CDC) mechanism to capture only the changed data from the source table and apply those changes to the target table incrementally.
Please ensure source table is append only and there are no updates and deletions done.