Sidhant07
Databricks Employee
Databricks Employee

Hi @Ovasheli ,

I believe the error message would be something like below.

Error:

com.databricks.sql.transaction.tahoe.DeltaUnsupportedOperationException: [DELTA_SOURCE_TABLE_IGNORE_CHANGES] Detected a data update (for example DELETE (Map(predicate - in the source table at version 166. This is currently not supported. If this is going to happen regularly and you are okay to skip changes, set the option 'skipChangeCommits' to 'true'. If you would like the data update to be reflected, please restart this query with a fresh checkpoint directory or do a full refresh if you are using DLT. If you need to handle these changes, please switch to MVs. The source table can be found at path abfss://oagdataproductdev@oagdevweussa.dfs.core.windows.net/__unitystorage/catalogs/a093d514-54ed-4254-a57a-c1b59f7d3efd/tables/2723f5df-35f5-4430-afcb-495298b8db4c.

As mentioned in the error message, using a new checkpoint or setting skipChangeCommits to true should help resolve the issue.

I handled similar issues in the past and we recommend this solutions to our users.