topic Re: Access Foreign Catalog using Python in Notebook in Get Started Discussions

Access Foreign Catalog using Python in Notebook

kashy — Sat, 26 Aug 2023 14:47:42 GMT

Hello - I have a foreign catalog which I can access fine in SQL. However, I can't access it from from python notebook.

i.e. this works just fine if I have notebook using a Pro SQL Warehouse

%sql
USE CATALOG <my_foreign_catalog_name>;
USE SCHEMA public;
SELECT * from bookings;

However, this, running on Shared Cluster 13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)

spark.sql("USE CATALOG <my_foreign_catalog_name>")
spark.sql("USE SCHEMA public")
display(spark.table("bookings"))

Gives me this error:

org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog 'my_foreign_catalog_name' plugin class not found: spark.sql.catalog.my_foreign_catalog_name is not defined

If I run:

display(spark.sql("SHOW CATALOGS"))

I can't see the foreign_catalogs I have created in Unity Catalog and it just shows me 'spark_catalog'

Any idea what needs tweaking?

Re: Access Foreign Catalog using Python in Notebook

Debayan — Thu, 07 Sep 2023 07:18:02 GMT

Hi, Are you using this in single user cluster?

Also, please tag @Debayan with your next response so that I will get notified.

Re: Access Foreign Catalog using Python in Notebook

kashy — Mon, 11 Sep 2023 09:24:53 GMT

thanks @Debayan

I resolved this - I has to remove this option on the cluster and it works now

Enable credential passthrough for user-level data access

Re: Access Foreign Catalog using Python in Notebook

MadhavaReddy03 — Thu, 26 Mar 2026 07:23:29 GMT

@Debayan , could you please help us how to resolve this issue with steps.