Material View to External Delta Table using sink api

[nkrom456]

Hi Team,

While executing the below code i am able to create the sink and my data is getting written into delta tables from materialized view.

import dlt
@Dlt.table(name = "employee_bronze3")
def create_table():
df = spark.read.table("dev.default.employee")
return df

@Dlt.table(name = "employee_silver3")
def create_table():
df = dlt.read("dev.default.employee_bronze3")
return df

dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)

@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
dlt.read_stream("dev.default.employee_silver3")
)

But if i already have a materialized view created from different dlt and execute the below statement i am getting the error The sink created is giving error due to the below reason : Cannot stream from Materialized View `dev`.`default`.`employee_silver3`. Streaming from Materialized Views is not supported but why am i not getting the error in the above statement.

dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)

@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
spark.readStream.table("dev.default.employee_silver3") , I have also tried with spark.read.table("dev.default.employee_silver3")
)

Can anyone please let me know why this is happening.

[Brahmareddy]

Hi nkrom456,

How are you doing today? as per my understanding, when you use dlt.read_stream() inside the same DLT pipeline, Databricks allows it to stream from that materialized view because everything is being managed within one pipeline — it understands the full flow. But when you try to use spark.readStream.table() to read a materialized view that was created by a different DLT pipeline, it throws an error because streaming from materialized views isn’t supported across pipelines. That’s a Databricks limitation right now. A better way to handle this is to have the first pipeline write to a regular Delta table instead of a materialized view, and then stream from that Delta table in your second pipeline. This keeps things simple and supported. Let me know if you want help updating the code!