@zed I went through these document links, and believe you're on spot here with respect to the limitation and exclusive setups. I don't see any option other than implementing a retry logic or reducing the possibilities of such conflicts, something like:

import time
import random
from delta.exceptions import ConcurrentAppendException

retry = 5
while retry > 0:
    try:
        # Your update statement on the delta table
        break
    except ConcurrentAppendException as e:
        retry -= 1
        delay = random.randrange(0, 20)
        time.sleep(delay)
        print(f"{retry} retries left, added delay {delay} seconds")
else:
    raise Exception("Update failed after multiple retries")

Concurrency control is very complex to handle. I'm not fully sure, but given the capabilities of LiquidClustering, it could potentially help with reducing the likelihood of conflicts, although at the end everything narrows down to having two whichever conflicting operations running concurrently and reducing the chances for having resulting data inconsistencies, batching updates togheter is another option, but also considered as a mitigation, I don't think there's a golden solution for this.

In your case (make sure to review it first):

    fe = FeatureEngineeringClient()
    retries = 5
    while retries > 0:
        try:
            fe.write_table(
                name="dbc_mlops_premium.taxi_example.trip_pickup_features",
                df=df_pickupzip_features,
                mode="merge"
            )
            break
        except Exception as e:
            retries -= 1
            if retries == 0:
                raise
            delay = 20  # You can adjust the delay as needed
            time.sleep(delay)
            print(f"Retrying... {retries} attempts left")