Dooley
Databricks Employee
Databricks Employee

@Long Tran​ , The best way to achieve this would be a work around. If you could sacrifice a row of code in your Table 1 or add a row of Nulls into Table 1 without causing problems for yourself further down your pipeline, I suggest you try this Retain manual deletes or updates - "You can manually delete or update the record from raw_user_table and do a refresh operation to recompute the downstream tables."

However, I want to note that if you are ingesting from a source that has no new data to ingest, the full refresh probably wont re-ingest the same data causing duplication. Try it out on a subsection of your data.

View solution in original post