@Sarah Guido :
It sounds like you are on the right track with importing the historical data into Delta Lake first and then converting the tables to Delta Live Tables using the CONVERT TO DELTA SQL statement. The error message you are seeing about a managed table already existing with that name is because you have already created a managed Delta table with that name in Delta Lake and now you are trying to create a Delta Live Table with the same name.
To avoid this error, you could try renaming the managed Delta table to something else before converting it to a Delta Live Table. You can do this using the ALTER TABLE SQL statement:
ALTER TABLE schema.table_name RENAME TO schema.new_table_name;
Once you have renamed the managed Delta table, you can use the CONVERT TO DELTA statement to create a Delta Live Table with the original name.
Regarding your other alternatives, using Photon acceleration for the initial import could be a good option if you have the resources to do so. This would allow you to import the data quickly and efficiently, and then you could start ingesting new data into DLT from that point forward.
Using a join or union in the DLT pipeline to combine the historical data in Delta Lake with new data ingested into DLT could also work, but as you mentioned, you would need to hard code the date at which DLT should start ingesting data. This could be problematic if you need to make changes to that date later on.
Overall, importing the historical data into Delta Lake first and then converting to Delta Live Tables is likely the best approach for your scenario. Renaming the managed Delta table before converting it to a Delta Live Table should allow you to avoid the error message you are seeing.
Hope this helps! Happy learning.
