topic Unable to use unity catalog in notebook in Data Engineering

Unable to use unity catalog in notebook

harraz — Wed, 31 May 2023 01:58:26 GMT

com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog 'uc-dev' plugin class not found: spark.sql.catalog.uc-dev is not defined
....

I get the above when I reference the unity catalog.

Below is the cell that produced this error

%sql
-- Resetting both catalog and schema
USE catalog `uc-dev`;
USE DATABASE uc-dev.myschema;
SELECT current_database(), current_catalog();

Any help is much appreaciated

Re: Unable to use unity catalog in notebook

karthik_p — Fri, 02 Jun 2023 14:33:43 GMT

@mohamed harraz can you please make sure if uc-dev catalog exists 2. please make sure cluster you are using is UC enabled (access mode: single/shared)

Re: Unable to use unity catalog in notebook

Anonymous — Wed, 07 Jun 2023 08:16:38 GMT

Hi @mohamed harraz

We haven't heard from you since the last response from @karthik p , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Re: Unable to use unity catalog in notebook

Tomas — Fri, 10 May 2024 00:48:28 GMT

I had the same error plugin class not found: spark.sql.catalog... is not defined immediately after attaching the workspace into Unity catalog.

The error was resolved by restarting SQL Warehouse.

It seems that if SQL Warehouse (or any cluster) is running at the time the workspace is attached into Unity catalog, then it shows this error.

Maybe it is a bug as the Unity catalog was visible in the Catalog menu directly, but all operations on top of this catalog were not working until the compute restart?