Hi,
We are trying to build and upsert logic for a Delta table for that we are writing a merge command between streaming dataframe and delta table dataframe. Please find the below code
merge_sql = f"""
Merge command come here
"""
spark.sql(merge_sql).writeStream \
.format("delta") \
.outputMode("append") \
.option("checkpointLocation", "bbbbb") \
.option("mergeSchema", "true") \
.option("path","aaaaaaa") \
.toTable(f"{target_catalog}.{target_schema}.silver_{sl_tablename}")
getting this below error. Please help us on how to resolve this error.
Error during merge operation on 'tablename': Queries with streaming sources must be executed with writeStream.start();
