Hi All,
In our project we are working on the DLT pipeline with the DLT tables as target running in continuous mode.
These tables are common for multiple countries, and we go live in batches for different countries.

So, every time a new change is requested by the business which demands a change in the metadata of the DLT table, we are updating the DLT table creation in the notebook and we are forced to run the pipeline in full refresh.

This is becoming a big concern as:

• Unless a table has data in its corresponding source system, we will not be able to reload the complete data as we do full-refresh and we might lose the history data which was already there in the current table prior to full-refresh
• The SLA gets impacted as the refresh takes huge time to rerun the complete 2 years of data sometimes even close to 7 hours

Is there a way that we can work with these DLT pipelines but at the same time, work from that point of time run even if it demands metadata updates? Please help