I am using 

@Dlt.table decorator to create a table which will store history for my tables.

My code works like this:

@Dlt.table(name="table name")

def target:

       Custom spark code to create history

Even though the spark code creates and prints history when i use it in normal notebook, but when I run it inside the pipeline, it does not create history, it just writes most recent record somehow.

Can someone tell me how exactly is this happening, and what I can improve, please!