cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Setting up a snowflake catalog via spark config next to unity catalog

Laurens
New Contributor II

Im trying to set up a connection to Iceberg on S3 via Snowflake as described https://medium.com/snowflake/how-to-integrate-databricks-with-snowflake-managed-iceberg-tables-7a889... and https://docs.snowflake.com/en/user-guide/tables-iceberg-catalog

On a cluster with Unity Catalog disabled this works fine and I can read from the tables without connecting to a snowflake warehouse. When Unity Catalog is enabled however, the `snowflake_catalog` is nowhere to be found.

Is there a way to setup a catalog via Spark config like described above while using Unity Catalog?

Thanks in advance!

2 REPLIES 2

Laurens
New Contributor II

Hi Kaniz,

Thanks for your elaborate reply! The iceberg data that we have to interact with uses Snowflake as catalog, not glue. Can we also use a catalog integration with Snowflake?

Thanks!

Laurens
New Contributor II

Hi @Retired_mod ,

We've been working on setting up Glue as catalog, which is working fine so far. However, Glue takes place of the hive_metastore, which appears to be a legacy way of setting this up. Is the way proposed here the recommended way to set it up, or is there a more "unity catalog" way of setting it up? https://docs.databricks.com/en/archive/external-metastores/aws-glue-metastore.html

Thanks in advance,

Laurens

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group