Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @IM_01,

You’re right to be suspicious here... a materialized view is not a safe or supported source for a streaming table, even if your pipeline currently runs without errors.

Streaming tables are built on Structured Streaming and assume the source is append‑only. New versions of the source table may only add rows. Existing rows are never changed. They only process new data since the last checkpoint and do not recompute history.

Materialized views are maintained by Enzyme with batch semantics. On each refresh (incremental or full) they are free to update or delete existing rows in their backing Delta table to keep the result correct. A full refresh can even truncate and recompute everything.

So if you do:

CREATE OR REFRESH STREAMING TABLE st AS
SELECT * FROM STREAM(my_mv);
you get a mismatch...
 
From the streaming engine’s perspective, my_mv looks like a Delta table whose history can be rewritten by MV refresh. As soon as the MV performs non‑append changes (corrections, late data, full refresh), you either hit a "non‑append source" style failure, or silently lose corrections if you’ve enabled options that skip non‑append commits.

The fact that your pipeline runs today usually just means the MV hasn’t yet done anything but pure appends. It’s not a pattern you should rely on.

Hope this answers your question.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

 
Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***