Iceberg interoperability between Databricks and external catalogs

[mukul1409]

I would like to understand the current approach for Iceberg interoperability in Databricks. Databricks supports Iceberg using Unity Catalog, but many teams also use Iceberg tables managed outside Databricks. Are there recommended patterns today for sharing or accessing Iceberg data across platforms. Is there any future plan to support external Iceberg REST based catalogs. Any guidance from the community would be helpful.

[iyashk-DB]

Hi @Mukul 

Databricks’ current direction is to use Unity Catalog as an open Iceberg catalog. UC exposes tables via the Iceberg REST Catalog API, so external engines (Spark, Flink, Trino, Snowflake, PyIceberg, etc.) can read and write UC-managed Iceberg tables while Databricks keeps governance and optimization.

If data lives outside Databricks (Glue/HMS/Snowflake), Databricks can read it via Lakehouse Federation (read-only). If data is written as Delta in Databricks but needs Iceberg consumers, UniForm (Iceberg reads) lets Iceberg clients read the same data without rewrites.

Databricks does not plan to write into external catalogs, but external engines can write into UC-managed Iceberg today. Overall, the model is: UC as the central catalog, open access via Iceberg REST, single copy of data, no duplication.

[emma_s · Accepted solution]

Hi, Databricks has just released into Public preview the ability to federate connect into Iceberg external tables. This would be the recommended way to access data into these tables and treat them as a foreign table. See docs here https://docs.databricks.com/aws/en/iceberg/

[Yogesh_Verma_]

Great