cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Setting up a snowflake catalog via spark config next to unity catalog

Laurens
New Contributor II

Im trying to set up a connection to Iceberg on S3 via Snowflake as described https://medium.com/snowflake/how-to-integrate-databricks-with-snowflake-managed-iceberg-tables-7a889... and https://docs.snowflake.com/en/user-guide/tables-iceberg-catalog

On a cluster with Unity Catalog disabled this works fine and I can read from the tables without connecting to a snowflake warehouse. When Unity Catalog is enabled however, the `snowflake_catalog` is nowhere to be found.

Is there a way to setup a catalog via Spark config like described above while using Unity Catalog?

Thanks in advance!

2 REPLIES 2

Laurens
New Contributor II

Hi Kaniz,

Thanks for your elaborate reply! The iceberg data that we have to interact with uses Snowflake as catalog, not glue. Can we also use a catalog integration with Snowflake?

Thanks!

Laurens
New Contributor II

Hi @Retired_mod ,

We've been working on setting up Glue as catalog, which is working fine so far. However, Glue takes place of the hive_metastore, which appears to be a legacy way of setting this up. Is the way proposed here the recommended way to set it up, or is there a more "unity catalog" way of setting it up? https://docs.databricks.com/en/archive/external-metastores/aws-glue-metastore.html

Thanks in advance,

Laurens

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group