cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Can multiple streams write to a Delta table at the same time?

User16826992666
Valued Contributor

Wondering if there any dangers to doing this, and if it's a best practice. I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

1 ACCEPTED SOLUTION

Accepted Solutions

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

View solution in original post

2 REPLIES 2

Ryan_Chynoweth
Esteemed Contributor

Delta would enforce this dual write using optimistic concurrency and would not have an issue generally accepting multiple writes. Please note that data quality could be an issue if there is duplicate data coming in from multiple streams of if there were conflicting transactions (two updates, deletes etc.).

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now