Issue with merge command between streaming dataframe and delta table

Venu_DE1 — Mon, 06 Nov 2023 12:55:03 GMT

Hi,

We are trying to build and upsert logic for a Delta table for that we are writing a merge command between streaming dataframe and delta table dataframe. Please find the below code

merge_sql = f"""

Merge command come here

"""

spark.sql(merge_sql).writeStream \
.format("delta") \
.outputMode("append") \
.option("checkpointLocation", "bbbbb") \
.option("mergeSchema", "true") \
.option("path","aaaaaaa") \
.toTable(f"{target_catalog}.{target_schema}.silver_{sl_tablename}")

getting this below error. Please help us on how to resolve this error.
Error during merge operation on 'tablename': Queries with streaming sources must be executed with writeStream.start();

topic Issue with merge command between streaming dataframe and delta table in Data Engineering

Issue with merge command between streaming dataframe and delta table