โ06-07-2021 11:09 AM
Does Spark Structured Streaming supports `OutputMode.Update` for Delta tables?
โ06-07-2021 11:11 AM
Nope, it's not supported, but you could use a MERGE statement inside of a forEachBatch streaming sync
Documentation on MERGE
https://docs.databricks.com/spark/latest/spark-sql/language-manual/delta-merge-into.html
Documentation for arbitrary streaming sinks
https://docs.databricks.com/spark/latest/structured-streaming/foreach.html
never-displayed
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!