Material View to External Delta Table using sink api

nkrom456 — Mon, 25 Aug 2025 14:17:38 GMT

Hi Team,

While executing the below code i am able to create the sink and my data is getting written into delta tables from materialized view.

import dlt
@Dlt.table(name = "employee_bronze3")
def create_table():
df = spark.read.table("dev.default.employee")
return df

@Dlt.table(name = "employee_silver3")
def create_table():
df = dlt.read("dev.default.employee_bronze3")
return df

dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)

@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
dlt.read_stream("dev.default.employee_silver3")
)

But if i already have a materialized view created from different dlt and execute the below statement i am getting the error The sink created is giving error due to the below reason : Cannot stream from Materialized View `dev`.`default`.`employee_silver3`. Streaming from Materialized Views is not supported but why am i not getting the error in the above statement.

dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)

@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
spark.readStream.table("dev.default.employee_silver3") , I have also tried with spark.read.table("dev.default.employee_silver3")
)

Can anyone please let me know why this is happening.

Re: Material View to External Delta Table using sink api

Brahmareddy — Sat, 06 Sep 2025 22:50:01 GMT

Hi nkrom456,

How are you doing today? as per my understanding, when you use dlt.read_stream() inside the same DLT pipeline, Databricks allows it to stream from that materialized view because everything is being managed within one pipeline — it understands the full flow. But when you try to use spark.readStream.table() to read a materialized view that was created by a different DLT pipeline, it throws an error because streaming from materialized views isn’t supported across pipelines. That’s a Databricks limitation right now. A better way to handle this is to have the first pipeline write to a regular Delta table instead of a materialized view, and then stream from that Delta table in your second pipeline. This keeps things simple and supported. Let me know if you want help updating the code!

topic Material View to External Delta Table using sink api in Data Engineering

Material View to External Delta Table using sink api

Re: Material View to External Delta Table using sink api