Does Spark Structured Streaming supports `OutputMode.Update` for Delta tables?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2021 11:09 AM
Does Spark Structured Streaming supports `OutputMode.Update` for Delta tables?
Labels:
- Labels:
-
Delta
-
Spark structured streaming
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2021 11:11 AM
Nope, it's not supported, but you could use a MERGE statement inside of a forEachBatch streaming sync
Documentation on MERGE
https://docs.databricks.com/spark/latest/spark-sql/language-manual/delta-merge-into.html
Documentation for arbitrary streaming sinks
https://docs.databricks.com/spark/latest/structured-streaming/foreach.html

