https://community.databricks.com/t5/data-engineering/delta-merge-drops-cached-variables/m-p/3819#M728 <P>Hi Everyone,</P><P></P><P>I have an ingestion script where I use the delta merge to update and append newly incoming data in dataframe format to an existing delta table.</P><P></P><P>I am experiencing an issue where all the variables that have been used previously loose their data after the execution of delta merge.</P><P></P><P>I have tried using cache and persist (both on disk and in memory) and they don't yield the expected results.</P><P></P><P>What might be the cause for such behaviour and how can I overcome it? </P><P></P><P>Thanks</P> Wed, 31 May 2023 10:33:13 GMT Altay 2023-05-31T10:33:13Z https://community.databricks.com/t5/data-engineering/delta-merge-drops-cached-variables/m-p/3819#M728 <P>Hi Everyone,</P><P></P><P>I have an ingestion script where I use the delta merge to update and append newly incoming data in dataframe format to an existing delta table.</P><P></P><P>I am experiencing an issue where all the variables that have been used previously loose their data after the execution of delta merge.</P><P></P><P>I have tried using cache and persist (both on disk and in memory) and they don't yield the expected results.</P><P></P><P>What might be the cause for such behaviour and how can I overcome it? </P><P></P><P>Thanks</P> Wed, 31 May 2023 10:33:13 GMT https://community.databricks.com/t5/data-engineering/delta-merge-drops-cached-variables/m-p/3819#M728 Altay 2023-05-31T10:33:13Z