Hello

I am working on building a pipeline using Medallion architecture. The source tables in this pipeline are overwritten each time the table is updated. In the bronze ingestion layer, I plan to append this new table to the current bronze table, adding an ingestion column to the source data to document time of ingestion. I also plan to remove any data older than the retention period dictated by my team.

I was wondering what is the difference between using Delta Lake Time Travel vs maintaining a bronze layer that contains historical data. Is there a major cost difference over choosing to use Time Travel over historical copies in bronze if the table is very big? Is there a processing time benefit over have a bronze layer that contains historical data over using time travel?