Delta Live Table automatic table removal and schema update

hayden_blair
New Contributor III

Hello, 

I made a delta live table workflow that created 3 streaming tables in unity catalog. I then removed the source code for the 3rd table from the workflow and reran. After about a week, the 3rd streaming table is no longer available in unity catalog and appears to have been automatically dropped by Databricks.

In this post@Retired_mod describes this as expected behavior.

I am now trying to add that 3rd table back into the workflow, but one of the columns has changed data types (double => long). I am getting the following error:

"[DELTA_MERGE_INCOMPATIBLE_DATATYPE] Failed to merge incompatible data types DoubleType and LongType"

The 3rd table has been dropped from unity catalog, but the schema of the original table still persists somewhere. How do I restore the old records of the original table using the new schema so that I can resume streaming with the new schema, without losing historical records?

Thank you.