3 weeks ago
Everytime I run parallel job it always failed with this error:
ConcurrentAppendException: Files were added to the root of the table by a concurrent update. Please try the operation again.
I did a lot of reseaches also create liquid clustering table and row level concurrency but it still failed.
Note: I can pass with retry method, but my boss doesn't like that approach. Does anyone can help me on this case?
Thanks you very much!
Please see detailt my table configuration in attachment picture.
3 weeks ago
Note: I tried both DBR 13.3.x and 14.3.x but still failed with same error
3 weeks ago
See this chart for a description of how row-level concurrency works. With row-level concurrency, concurrent merge operations still cannot safely modify the same row. Without row-level concurrency, concurrenty merge operations cannot safely modify the same partition.
Here are two strategies for handling this (there may be more):
Please be aware that target tables with identity columns enabled do not support concurrent operations, regardless of disjoint predicates.
3 weeks ago - last edited 3 weeks ago
Hi @cgrant,
Thanks for your checking, Could you please help me review on this?I also did the disjoint predicate here is my condition for merge,
3 weeks ago
What do the other merge commands look like?
3 weeks ago
2 weeks ago
Hi @cgrant,
Do you have any update on this? Thanks and very appreciate for your help!
2 weeks ago
Thanks for sharing. In the original screenshots I've noticed that you've set delta.isolationLevel to Serializable, which is the strongest (and most strict) level. Please try WriteSerializable, which is the default level.
2 weeks ago
Hi @cgrant,
Yes, I just tried to change delta.isolationLevel to WriteSerilizable but still failed, please view my screentshoot
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group