cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta live table is inserting data multiple times

zero234
New Contributor III

So I have created a delta live table 

Which uses spark.sql() to execute a query 

And uses df.write.mode(append).insert into

To insert  data into the respective table And at the end i return a dumy table 

Since this was the requirement So now I have also implemented count checks but still it is inserting the data 5-7 times I can check the same in table history where the pipline triggers and executes it for 7 times even though it has checked to insure if data is present do not insert it is still inserting Can someone help me debug this And how is initialisation doen in delta live table 

 

 

1 REPLY 1

jose_gonzalez
Moderator
Moderator

whats your source? your sink is a delta table correct? how do you verify that there are no inserts happening? 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group