Prevent Duplicate Entries to enter to delta lake Storage

[User16826994223]

I have a data frame and I write this data frame to adls table, next day I get an updated data frame which has some records from the past also and i want to update the delta table without creating duplicate

[User16826994223]

This is a task for Merge command - you define condition for merge (your unique column) and then actions.

MERGE INTO target
USING  src
ON target.column = source.column
WHEN MATCHED THEN
  UPDATE SET *
WHEN NOT MATCHED
  THEN INSERT *

could be your dataframe registered as temporary view):