cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks delta tables and Iceberg

Sam500
New Contributor III

Hi experts ,now databricks for support for Iceberg open table format conversion yet not fully though.Databricks calls this Delta Lake UniForm (Universal Format). When tentatively Databricks can support fully open table format Iceberg, Hudi etc.Is there any future roadmap in this direction.Thank you

1 ACCEPTED SOLUTION

Accepted Solutions

DoTA
Contributor III

Worth separating "reading Iceberg from Delta" from "Databricks fully adopting Iceberg as a first-class table format" - those are two different things and Databricks has been moving on both fronts.

 

Today: UniForm (Universal Format) makes a Delta table also readable by Iceberg (and, in preview, Hudi) clients by generating the corresponding metadata (e.g. Iceberg manifests/manifest lists) alongside the Delta log - no data rewrite needed. It's read-compatibility, not a native Iceberg table; the table is still fundamentally Delta under the hood.

 

Roadmap direction: since acquiring Tabular (the company founded by the original Iceberg creators) in mid-2024, Databricks has been explicit about converging Delta and Iceberg rather than treating them as permanently separate formats - the stated goal is full interoperability so the format you use matters less, and both communities benefit from shared improvements (this is also part of the motivation behind Delta Kernel and the Iceberg V3 spec work). Unity Catalog has also added Iceberg REST Catalog support, so external Iceberg-only engines can read Unity Catalog-managed tables through the standard Iceberg REST API without needing UniForm-specific tooling.

 

There's no public commitment I'm aware of to natively support Hudi as a writable/managed format the way Delta and (increasingly) Iceberg are heading - Hudi support today is UniForm read-compat only.

 

For the latest specifics (GA status, which regions/runtimes), the release notes and the Unity Catalog + Delta Lake docs are the most reliable source since this area is moving fast - the broad direction (converging toward open interoperability rather than a Delta-only future) is clear, but exact feature/GA timing changes release to release.

View solution in original post

1 REPLY 1

DoTA
Contributor III

Worth separating "reading Iceberg from Delta" from "Databricks fully adopting Iceberg as a first-class table format" - those are two different things and Databricks has been moving on both fronts.

 

Today: UniForm (Universal Format) makes a Delta table also readable by Iceberg (and, in preview, Hudi) clients by generating the corresponding metadata (e.g. Iceberg manifests/manifest lists) alongside the Delta log - no data rewrite needed. It's read-compatibility, not a native Iceberg table; the table is still fundamentally Delta under the hood.

 

Roadmap direction: since acquiring Tabular (the company founded by the original Iceberg creators) in mid-2024, Databricks has been explicit about converging Delta and Iceberg rather than treating them as permanently separate formats - the stated goal is full interoperability so the format you use matters less, and both communities benefit from shared improvements (this is also part of the motivation behind Delta Kernel and the Iceberg V3 spec work). Unity Catalog has also added Iceberg REST Catalog support, so external Iceberg-only engines can read Unity Catalog-managed tables through the standard Iceberg REST API without needing UniForm-specific tooling.

 

There's no public commitment I'm aware of to natively support Hudi as a writable/managed format the way Delta and (increasingly) Iceberg are heading - Hudi support today is UniForm read-compat only.

 

For the latest specifics (GA status, which regions/runtimes), the release notes and the Unity Catalog + Delta Lake docs are the most reliable source since this area is moving fast - the broad direction (converging toward open interoperability rather than a Delta-only future) is clear, but exact feature/GA timing changes release to release.