Here are your options:

1. Best fix: stop using partitioned target tables for concurrent MERGE Partitioned Delta tables do not support row-level concurrency. If you can, move the target to unpartitioned + Liquid Clustering + deletion vectors. For MERGE, use DBR 14.3 LTS+ (or 14.2 with Photon). This is the cleanest way to reduce concurrent merge conflicts.

2. If you must keep partitions, make the MERGE predicate fully explicit Don’t just join on PK. Include the target partition filters in the MERGE condition so each job reads only its own partition slice; otherwise separate jobs can still scan overlapping data and conflict.

3. If jobs can touch the same rows, serialize writes Even with row-level concurrency, conflicts can still happen when operations read/modify the same rows, and with high concurrency some conflicts still occur. In that case, use a staging table + one downstream MERGE job (or queue by business key) instead of many direct concurrent MERGEs.

Recommendation: Option 1 if you can change the table design. If not, use Option 2, and if the same keys can arrive together, add Option 3.