Unable to read from or write to Snowflake Open Catalog via Databricks

[Sunil_Patidar]

I have Snowflake Iceberg tables whose metadata is stored in Snowflake Open Catalog. I am trying to read these tables from the Open Catalog and write back to the Open Catalog using Databricks.

I have explored the available documentation but haven’t been able to find clear guidance on how to achieve this. One Snowflake document mentions that Apache Spark can read from and write to Snowflake Open Catalog:
https://docs.snowflake.com/en/user-guide/opencatalog/register-service-connection#examples
Based on this, Iam assuming that if Apache Spark supports read/write operations with Snowflake Open Catalog, Databricks (which is Spark-based) should also be able to do the same.

My main goal is to achieve interoperability between Snowflake and Databricks using Snowflake Open Catalog, which is compatible with the Iceberg REST Catalog.
Could someone please clarify:
Whether Databricks officially supports reading from and writing to Snowflake Open Catalog
If so, are there any reference architectures, examples, or configuration steps available?
Any guidance or pointers to the correct documentation would be greatly appreciated.

Thanks in advance!

[Louis_Frolio]

Greetings @Sunil_Patidar ,  Databricks and Snowflake can interoperate cleanly around Iceberg today — but how you do it matters.

At a high level, interoperability works because both platforms meet at Apache Iceberg and the Iceberg REST Catalog API.

What works today

• Snowflake can read (GA) and write (Public Preview) Unity Catalog–managed Iceberg tables using Databricks’ Iceberg REST Catalog implementation in Unity Catalog.

• Databricks can discover and read Iceberg tables managed by Snowflake Horizon Catalog using Unity Catalog Iceberg catalog federation (Public Preview, requires Databricks Runtime 16.4 LTS+).

• Databricks can also connect directly to Snowflake Open Catalog using the standard Apache Iceberg REST client (Spark catalog). This supports full read/write and is the most flexible option today.

Important nuance

Write-back to Snowflake Horizon via Unity Catalog federation is not explicitly documented as supported in the current Public Preview. If you need true bidirectional writes between Databricks and Snowflake-managed Iceberg tables, use the direct Spark + Iceberg REST approach rather than UC federation.

Recommended patterns

If Snowflake needs access to Databricks-managed tables

→ Use Unity Catalog’s Iceberg REST APIs with Snowflake’s REST catalog integration.

If Databricks needs read access to Snowflake Horizon tables

→ Use Unity Catalog Iceberg catalog federation (DBR 16.4 LTS+).

If you need full read/write interoperability

→ Configure Databricks as an Apache Iceberg Spark client pointing directly at Snowflake Open Catalog via the Iceberg REST API.

Key things to watch

• Use fully qualified catalog names in UC-enabled workspaces (don’t replace spark_catalog).

• Match Iceberg runtime versions to your Databricks Runtime.

• Snowflake Open Catalog uses catalog-vended, short-lived credentials via the Iceberg REST API — no long-lived cloud credentials required.

Bottom line

Iceberg interoperability between Databricks and Snowflake is real and usable today, but federation is currently best suited for discovery and read access. For bidirectional data movement, go direct via Iceberg REST.

Hope that helps, Louis.

[Sunil_Patidar]

Hi @Louis_Frolio  — thanks for sharing the detailed information.
I am working on enabling bidirectional data movement using Iceberg REST APIs. I am able to successfully read and write to the Snowflake Open Catalog from the Snowflake console. However, when I configure the Open Catalog in Databricks, I am unable to read from — or write to — the Open Catalog from Databricks.

When I run the configuration below, the Databricks/Spark session continues to resolve against the Databricks default catalog instead of the Snowflake Open Catalog. The session runs without any errors, but it never switches to the expected catalog.

I have applied the following properties at the cluster level:

spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:1.4.1,org.apache.iceberg:iceberg-gcp-bundle:1.5.2
spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
spark.sql.defaultCatalog opencatalog
spark.sql.catalog.opencatalog org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.opencatalog.type rest
spark.sql.catalog.opencatalog.uri https://xxxxxxx.snowflakecomputing.com/polaris/api/catalog
spark.sql.catalog.opencatalog.header.X-Iceberg-Access-Delegation vended-credentials
spark.sql.catalog.opencatalog.credential xxxxxx=:xxxxxxx
spark.sql.catalog.opencatalog.warehouse snowflake_internal_catalog -- My catalog
spark.sql.catalog.opencatalog.scope PRINCIPAL_ROLE:snowflake_principal_role

I would appreciate any guidance on what I might be missing — particularly around catalog resolution or additional configuration required on the Databricks side.

[mukul1409]

Databricks does not currently provide official support to read from or write to Snowflake Open Catalog. Although Snowflake Open Catalog is compatible with the Iceberg REST catalog and open source Spark can work with it, this integration is not supported in Databricks managed runtimes. Databricks officially supports Iceberg through Unity Catalog and documented integrations only. Using Snowflake Open Catalog directly from Databricks would require unsupported configurations and is not recommended for production. For interoperability today the supported options are Unity Catalog based Iceberg sharing or data exchange through cloud storage or managed data sharing solutions.