cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta Live Table - Flow detected an update or delete to one or more rows in the source table

manish1987c
New Contributor III

I have create a pipeline where i am ingesting the data from bronze to silver and using SCD 1, however when i am trying to create gold table as dlt it is giving me error as "Flow 'user_silver' has FAILED fatally. An error occurred because we detected an update or delete to one or more rows in the source table."

#DLT  #DeltaLiveTables

manish1987c_0-1718341166099.png

 

 

Flow 'gold_table_test' has FAILED fatally. An error occurred because we detected an update or delete to one or more rows in the source table. Streaming tables may only use append-only streaming sources. If you expect to delete or update rows to the source table in the future, please convert table gold_table_test to a materialized view instead of a streaming table. If you are fine to skip changes, set the option 'skipChangeCommits' to 'true'. If you only have a one-time change in the source table and will only have append-only commits from now on, you can perform a full refresh to table gold_table_test to resolve the issue. A Full Refresh will attempt to clear all data from table gold_table_test and then load all data from the streaming source. The non-append change can be found at version 27. Operation: MERGE Username: mk1987c@outlook.com Source table name: silver_load_test 

 

manish1987c_1-1718341206991.png

 

 

 

 

 

 

 

 

2 REPLIES 2

shan_chandra
Esteemed Contributor
Esteemed Contributor

@manish1987c -The Streaming does not handle input that is not an append. you can set skipChangeCommits to true 

Can you please share some sample code... Like how to use it.. 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!