cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta concurrency write Issue

User16826994223
Honored Contributor III

What is concurrent issue in delta, If at a time if we try to write same delta table , it some times fail , how to mitigate that

1 REPLY 1

Ryan_Chynoweth
Honored Contributor III
  1. Read: Reads (if needed) the latest available version of the table to identify which files need to be modified (that is, rewritten).
    1. Write: Stages all the changes by writing new data files.
    2. Validate and commit: Before committing the changes, checks whether the proposed changes conflict with any other changes that may have been concurrently committed since the snapshot that was read. If there are no conflicts, all the staged changes are committed as a new versioned snapshot, and the write operation succeeds. However, if there are conflicts, the write operation fails with a concurrent modification exception rather than corrupting the table as would happen with the write operation on a Parquet table.

Read more about delta lake concurrency here.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.