cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Can multiple streams write to a Delta table at the same time?

User16826992666
Valued Contributor

Wondering if there any dangers to doing this, and if it's a best practice. I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

1 ACCEPTED SOLUTION

Accepted Solutions

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

View solution in original post

2 REPLIES 2

Ryan_Chynoweth
Esteemed Contributor

Delta would enforce this dual write using optimistic concurrency and would not have an issue generally accepting multiple writes. Please note that data quality could be an issue if there is duplicate data coming in from multiple streams of if there were conflicting transactions (two updates, deletes etc.).

sajith_appukutt
Honored Contributor II

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group