Does Materialized views gets completely recalculated when we trigger DLT pipeline? Can't we start from where it left?In continuous mode of DLT pipeline Materialized view tries to optimizes the updates and computes the data?
Can apply_changes feature deal with deleted records in incoming source Files?By delete I mean record is being removed (Not a soft delete with Flag).If not, how to automate with deleting records from Bronze Streaming table based on source Files.
Hi Vinodh,Seems like DLT cannot handle it on its own. I think of a solution which goes like this.1) Maintain the copy of 200k files in a location (copied path). DLT should point to this copied path.2)If there is any change in the incoming file. Run a...
APPLY CHANGES can do upserts but I have doubts regarding deleting records by key.Does APPLY CHANGES can deletes record from bronze streaming if key is not present in source delta files.
