cgrant
Databricks Employee
Databricks Employee

Here are some strategies for debugging this:

  • Before you perform each merge, write your source dataframe out as a table, and include the target table's version in the table's name
  • If possible, enable the change data feed on your table so as to see changes for each version
  • The next time you witness this behavior, cross-reference the source table that you've written out with the table's changes.

If you can provide a simple reproduction of the issue, you can reply to this thread or reach out to Databricks support for help