cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Can multiple streams write to a Delta table at the same time?

User16826992666
Valued Contributor

Wondering if there any dangers to doing this, and if it's a best practice. I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

1 ACCEPTED SOLUTION

Accepted Solutions

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

View solution in original post

2 REPLIES 2

Ryan_Chynoweth
Honored Contributor III

Delta would enforce this dual write using optimistic concurrency and would not have an issue generally accepting multiple writes. Please note that data quality could be an issue if there is duplicate data coming in from multiple streams of if there were conflicting transactions (two updates, deletes etc.).

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.