Cross-engine Iceberg V3 compatibility issue- Databricks DELETE deletion vectors not readable by ext

[SRAVYA_TERADATA]

Hi Databricks Support Team,

We are raising this ticket to report a cross-engine compatibility issue between Databricks and Teradata when performing DELETE operations on an Iceberg V3 table.

---

ENVIRONMENT DETAILS
-------------------
Databricks Runtime Version : 17.3 LTS (Apache Spark 4.0.0, Scala 2.13)
Iceberg Table Format : V3
External Engine : Teradata (OTF — Open Table Format reader)
Storage : AWS S3
Catalog : Iceberg REST Catalog (Unity Catalog)

PROBLEM DESCRIPTION
-------------------
We have an Iceberg V3 table created from Teradata with the following properties:

'format-version' = '3'
'write.delete.mode' = 'merge-on-read'
'write.update.mode' = 'merge-on-read'

When DELETE operations are performed from Databricks on this table, Databricks generates deletion vectors as .bin files in the S3 location instead of Puffin format (.puffin) as defined in the Iceberg V3 specification.

Subsequent SELECT queries from Teradata OTF fail with the following error:

"Failure 6325 OTF error: Failed to read Puffin file: Failed to fetch manifest file object"

Teradata's OTF reader supports only the Puffin file format for deletion vectors as per the Iceberg V3 spec. It cannot read the .bin deletion vector format that Databricks generates.

---

STEPS TO REPRODUCE
------------------
1. Create Iceberg V3 table from Teradata with merge-on-read delete mode
2. Insert data from Teradata
3. Perform DELETE from Databricks:
DELETE FROM table WHERE mod(i1, 2) = 0;
DELETE FROM table WHERE i1 BETWEEN 5 AND 10;
4. Run SELECT from Teradata — fails with error 6325
5. Check S3 location — only .bin files generated, no .puffin files

---

QUESTIONS
---------
1. Is there a configuration or table property in Databricks DBR 17.3 to force deletion vectors to be written in Puffin format instead of .bin format for Iceberg V3 tables?

2. Is this a known limitation for cross-engine Iceberg V3 interoperability?

3. Is there a planned fix or roadmap item to support Puffin-format deletion vectors in Databricks for Iceberg V3 tables?

[Louis_Frolio]

Hi @SRAVYA_TERADATA , nice writeup. The environment details and repro steps make this easy to follow.

First, some context on what you're seeing. The Iceberg v3 spec defines a deletion vector as a deletion-vector-v1 blob stored in a Puffin file, and the delete manifest tracks each DV by referenced_data_file, content_offset, and content_size_in_bytes. That last part matters. The v3 blob encoding was deliberately aligned with Delta Lake's deletion vector encoding so both formats can share one copy of the delete data, and the manifest offsets let a reader pull the bitmap straight out of the containing file without depending on the file extension. Databricks writes its deletion vectors with that shared encoding, which is why you see .bin files instead of .puffin. A reader that validates the classic Puffin container (or keys off the extension) will fail on them, and that appears to be what Teradata's OTF reader does. Both engines are reading the same spec but disagree about the container, and that's the gap you've hit.

Before treating the .bin suffix as the whole story, it would help to confirm what the metadata actually says. If you can share these with bucket names, account identifiers, and any customer data redacted, the picture gets much clearer:

• format-version and the current snapshot ID
• The delete-manifest entries for the affected files: content, file_format, referenced_data_file, content_offset, and content_size_in_bytes
• Whether the referenced object is a Puffin container holding a deletion-vector-v1 blob or a standalone .bin object

That distinction tells us whether this is a nonstandard layout, a reader limitation on the Teradata side, or a version-specific mismatch.

On your three questions:

1. I can't find a table property in DBR 17.3 that forces a .puffin container, and I don't believe one exists. Also, write.delete.mode and write.update.mode control merge-on-read versus copy-on-write behavior; they aren't encoding switches, so don't expect them to change the container. What you can control is whether DVs get written at all. Two things to try on a scratch table: disable deletion vectors so deletes run copy-on-write (ALTER TABLE ... SET TBLPROPERTIES ('iceberg.enableDeletionVectors' = 'false')), or keep DVs for write performance and run REORG TABLE ... APPLY (PURGE) before Teradata reads, which materializes the deletes and drops the DV files. I haven't tested either path on a Teradata-created v3 table, so verify before you rely on it.

2. It isn't listed in the documented v3 limitations, so I can't point you at an official statement either way. Cross-engine v3 DV interop is still young. Also, the AWS docs currently list DBR 18 and above for reading and writing managed tables with Iceberg v3, and the feature is still in preview, so retest on the latest runtime (and the latest Teradata OTF release) if you can. The result may be specific to DBR 17.3.

3. Roadmap questions won't get a reliable answer on the forum, and I can't confirm a public roadmap item for Puffin-container output. File a support case with both vendors and include the redacted manifests plus your exact DBR version, Iceberg reader version, REST Catalog details, and Teradata OTF version. The key question for Databricks Support: does the tested runtime write spec-compliant deletion-vector-v1 Puffin blobs when deleting from a table served through the Unity Catalog Iceberg REST Catalog?

Until both engines verify against the same DV layout, the safest interim pattern is to route DELETE, UPDATE, and MERGE through the engine that produces the format Teradata can read, or use a copy-on-write workflow that rewrites the affected data files, then validate the resulting snapshot from both sides.

One sanity check as well: the error text also says it failed to fetch the object. Confirm Teradata can actually reach the DV path in S3 (credential scope, listing, GET). If the fetch succeeds and the read still fails, then it's the container check.

References:

• Apache Iceberg table spec, deletion vectors: https://iceberg.apache.org/spec/#deletion-vectors
• Apache Iceberg Puffin spec: https://iceberg.apache.org/puffin-spec/
• Use Apache Iceberg v3 features (Databricks): https://docs.databricks.com/aws/en/iceberg/iceberg-v3
• Deletion vectors in Databricks: https://docs.databricks.com/aws/en/tables/features/deletion-vectors
• Databricks blog, Apache Iceberg v3: https://www.databricks.com/blog/apache-icebergtm-v3-moving-ecosystem-towards-unification

Regards, Louis

[masonreed11]

This appears to be a cross-engine Iceberg V3 compatibility issue. If Databricks writes .bin deletion vectors while Teradata only supports Puffin, the key question is whether Puffin output can be enabled in DBR 17.3. I’d ask Databricks to confirm current support and any planned fix, and use a compatible workaround until then.