Historical Migration of Data from Delta Tables to Delta Live Tables

nalindhar
New Contributor II

Our team is planning to migrate to the Delta Live Tables (DLT) framework for data ingestion. We currently have Delta tables populated with several years of data from ingested files and wish to avoid re-ingesting these files. What is the best approach for transitioning this historical data into Delta Live Tables? Additionally, since DLT enforces that a single pipeline manages writes to a Delta Live Table and maintains an event log for proper commits, how can we efficiently handle historical loads, which involve straightforward inserts, while supporting daily loads that require different ingestion logic.