Can multiple streams write to a Delta table at the same time?

User16826992666
Databricks Employee
Databricks Employee

‎06-16-2021 08:27 PM

Wondering if there any dangers to doing this, and if it's a best practice. I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

0 Kudos

Ryan_Chynoweth
Databricks Employee
Databricks Employee

‎06-17-2021 05:05 PM

Delta would enforce this dual write using optimistic concurrency and would not have an issue generally accepting multiple writes. Please note that data quality could be an issue if there is duplicate data coming in from multiple streams of if there were conflicting transactions (two updates, deletes etc.).

0 Kudos

sajith_appukutt
Databricks Employee
Databricks Employee

‎06-17-2021 10:51 PM

>Can multiple streams write to a Delta table at the same time?

Yes delta uses optimistic concurrency control and configurable isolation levels

>I'm concerned there could be conflicts but I'm not sure how Delta would handle it.

Write operations can result in conflicts depending on the isolation level and could raise an exception. For more details on the types of exceptions and how to handle them - see https://docs.databricks.com/delta/concurrency-control.html#conflict-exceptions

View solution in original post

0 Kudos