- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-23-2026 12:46 PM
Hi @AanchalSoni,
I can’t see the full history of your notebook, so I’m not sure of the exact cause. But the behaviour strongly suggests that an earlier version of the stream used complete mode against the same checkpointLocation, and that configuration is what’s causing the error now.
Your current call is display(accounts_df, output_mode="append", checkpointLocation=".../Checkpoint/")
The error, however, says Invalid streaming output mode: complete. This output mode is not supported for no streaming aggregations...
For a non‑aggregated stream in append mode, Spark wouldn’t complain about complete unless it was reading that mode from somewhere else. In Structured Streaming, the only source of this is the checkpoint metadata. The checkpoint stores the original query plan, including the output mode. When you reuse the same checkpoint path with a changed query (no agg + append), Spark detects a mismatch between the stored configuration (complete) and the new query, and throws STREAMING_OUTPUT_MODE.UNSUPPORTED_OPERATION. When you delete the checkpoint, you erase that metadata, and the stream starts clean, which is why it fixes the issue.
The recommendation is not to reuse a checkpoint path across different query shapes or output modes. Give each logical stream (and each output mode) its own checkpointLocation.
Can you confirm whether you have any other processing steps before the cell shown in the snapshot?
If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***