Interesting observation about the INSERT before UPDATE pattern, that’s actually a useful clue for narrowing this down.
The iceberg-version=unspecified is worth investigating separately - maybe it for some reason ignoring your jars, but this only the hypothesis. On the duplicates though: if this were purely a format-level issue with position deletes not being supported, you’d expect duplicates on every UPDATE. The fact that it’s intermittent and specifically tied to a preceding INSERT points more toward a concurrency or snapshot isolation issue: the UPDATE may not be seeing the correct snapshot state after the INSERT commits, depending on how Glue catalog resolves the latest metadata between operations.
This would explain the inconsistency: it’s timing and snapshot visibility dependent, not a deterministic failure.
Worth checking: what Iceberg spec version is your table using? SELECT * FROM your_table.snapshots and look at the summary field. And have you tried using a separate catalog namespace like spark.sql.catalog.glue instead of overriding spark_catalog? With UC enabled, overriding spark_catalog is a known source of catalog resolution conflicts that could cause exactly this kind of intermittent snapshot visibility problem.
