Hi Team,
While executing the below code i am able to create the sink and my data is getting written into delta tables from materialized view.
import dlt
@Dlt.table(name = "employee_bronze3")
def create_table():
df = spark.read.table("dev.default.employee")
return df
@Dlt.table(name = "employee_silver3")
def create_table():
df = dlt.read("dev.default.employee_bronze3")
return df
dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)
@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
dlt.read_stream("dev.default.employee_silver3")
)
But if i already have a materialized view created from different dlt and execute the below statement i am getting the error The sink created is giving error due to the below reason : Cannot stream from Materialized View `dev`.`default`.`employee_silver3`. Streaming from Materialized Views is not supported but why am i not getting the error in the above statement.
dlt.create_sink(
name = "delta_sink",
format = "delta",
options = {"tableName": "dev.default.employee_test"}
)
@Dlt.append_flow(name = "delta_sink_flow", target="delta_sink")
def delta_sink_flow():
return(
spark.readStream.table("dev.default.employee_silver3") , I have also tried with spark.read.table("dev.default.employee_silver3")
)
Can anyone please let me know why this is happening.