Concurrent issue on delta lake insert update

[Dhanushn]

Hey team! I need your help on delta lake let me explain the scenario of mine.

Scenario: ive a table in delta lake and ive 2 databricks workflows running parallely which has insert and update tasks to do.
My delta table is partitioned with country code

My code for insert is
Df.write.mode(“append”).saveAsTable(table_name)

For update using merge

deltaTable.as("t").merge(
Df.as("s"),
"s.user_id = t.user_id")
.whenMatched().update(“t.col”:”s.col”….)

Im getting concurrent error when my job running parallely.

#databricks #deltalake 

[Takuya-Omi]

Hi, @Dhanushn 

In response to your question, the community contains the following information:

https://community.databricks.com/t5/community-platform-discussions/concurrent-update-to-delta-throws...

https://kb.databricks.com/en_US/delta/insert-operation-fails-while-trying-to-execute-multiple-concur...

Additionally, you may have already seen it, but here’s a link to the relevant section in the official documentation.

https://docs.databricks.com/en/optimizations/isolation-level.html#concurrentappendexception

I hope this information is helpful to you!