ignoreDeletes' option with Delta Live Table streaming source
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-01-2022 01:56 PM
We have a delta streaming source in our delta live table pipelines that may have data deleted from time to time.
The error message is pretty self explanatory:
...from streaming source at version 191. This is currently not supported. If you'd like to ignore deletes, set the option 'ignoreDeletes' to 'true'.
What's not clear is how to set this option. This is what we have now but it's not producing the desired results. The desired result being new data is read and deletes are ignored.
SET pipelines.ignoreDeletes = true;
CREATE OR REFRESH STREAMING LIVE TABLE...How should this option be set in a delta live table?