Does Unity Catalog support Iceberg?

[CT123456]

Hi,

I came across this post on querying an AWS Glue table in S3 from Databricks using Unity Catalog: https://medium.com/@sidnahar13/aws-glue-meets-unity-catalog-effortless-data-governance-querying-in-d.... The post says support for Iceberg is coming soon in public preview. Will the support for Iceberg allow us to query S3 Tables (AWS managed Iceberg tables) from Databricks, or this support is already available?

Thanks,

Calvin

[BigRoux · Accepted solution]

• Databricks already supports querying AWS Glue-managed Iceberg tables stored in S3 by integrating with the AWS Glue Iceberg REST Catalog.
• Unity Catalog in Databricks supports Iceberg reads on Delta tables (UniForm), enhancing interoperability with Iceberg clients.
• The public preview announcements refer to expanding and improving these capabilities, not the initial availability.
• For querying AWS-managed Iceberg tables (including S3 Tables), you need to configure Databricks clusters properly with the required libraries and Spark configurations.

Therefore, you can currently query AWS Glue Iceberg tables in S3 from Databricks.  Check out this blog post.

If you want to set this up, focus on configuring your Databricks cluster with the Glue Iceberg REST Catalog endpoint and ensure you have the right runtime (15.4 LTS or later) and dependencies installed, as shown in AWS documentation and blog posts.

Hope this helps, Lou.

[CT123456]

Thanks BigRoux. I had seen the AWS blog post that you mentioned. On the following:

• For querying AWS-managed Iceberg tables (including S3 Tables), you need to configure Databricks clusters properly with the required libraries and Spark configurations.

Does it mean I would have to use Spark to access an S3 Table from Databricks? Are there non-Spark ways to do the same?

[BigRoux]

As of now, Databricks does not support non-Spark methods (such as direct REST API calls, JDBC/ODBC connections outside Spark, or other query engines within Databricks) to query S3 Tables directly.

Unity Catalog and Databricks’ native cataloging focus primarily on Delta Lake tables and Spark SQL workloads. While Unity Catalog is evolving to support Iceberg via REST Catalog APIs, this is mostly to enable interoperability and external access rather than providing non-Spark query options inside Databricks.

Hope this helps, Lou.