topic Re: OPTIMIZE throws an error after doing MERGE on the table in Data Engineering https://community.databricks.com/t5/data-engineering/optimize-throws-an-error-after-doing-merge-on-the-table/m-p/29959#M21646 <UL><LI>You can try to change isolation level:</LI></UL><P><A href="https://docs.microsoft.com/en-us/azure/databricks/delta/optimizations/isolation-level" target="test_blank">https://docs.microsoft.com/en-us/azure/databricks/delta/optimizations/isolation-level</A></P><UL><LI>In merge is good to specify all partitions in merge conditions.</LI><LI>It can also happen when script is running concurrently.</LI></UL> Wed, 02 Feb 2022 14:37:00 GMT Hubert-Dudek 2022-02-02T14:37:00Z OPTIMIZE throws an error after doing MERGE on the table https://community.databricks.com/t5/data-engineering/optimize-throws-an-error-after-doing-merge-on-the-table/m-p/29957#M21644 <P>I have a table on which I do upsert i.e. </P><PRE><CODE>MERGE INTO table_name ...</CODE></PRE><P>After which I run </P><PRE><CODE>OPTIMIZE table_name</CODE></PRE><P>Which throws an error</P><PRE><CODE>java.util.concurrent.ExecutionException: io.delta.exceptions.ConcurrentDeleteReadException: This transaction attempted to read one or more files that were deleted (for example part-00000-50e8fcea-1314-445b-a4fd-a7b61a9bf02c-c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.</CODE></PRE><P>Not sure, what's happening as I am not deleting any files. Is there a way to fix this?</P> Wed, 02 Feb 2022 02:58:10 GMT https://community.databricks.com/t5/data-engineering/optimize-throws-an-error-after-doing-merge-on-the-table/m-p/29957#M21644 Constantine 2022-02-02T02:58:10Z Re: OPTIMIZE throws an error after doing MERGE on the table https://community.databricks.com/t5/data-engineering/optimize-throws-an-error-after-doing-merge-on-the-table/m-p/29959#M21646 <UL><LI>You can try to change isolation level:</LI></UL><P><A href="https://docs.microsoft.com/en-us/azure/databricks/delta/optimizations/isolation-level" target="test_blank">https://docs.microsoft.com/en-us/azure/databricks/delta/optimizations/isolation-level</A></P><UL><LI>In merge is good to specify all partitions in merge conditions.</LI><LI>It can also happen when script is running concurrently.</LI></UL> Wed, 02 Feb 2022 14:37:00 GMT https://community.databricks.com/t5/data-engineering/optimize-throws-an-error-after-doing-merge-on-the-table/m-p/29959#M21646 Hubert-Dudek 2022-02-02T14:37:00Z