Is enabling Unity catalog Mandatory to run the LLM Chatbot With RAG, DBRX demo?

[Abhishek_blue]

Hi! I am new to databricks. I was trying to run the RAG demo notebooks in databricks (@ Link to the demo: Deploy Your LLM Chatbot With Retrieval Augmented Generation (RAG), Foundation Models and Vector Sear...), in which the 00-init notebook in the resources folder did not run. The error: unity catalog not enabled.

[UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster. SQLSTATE: 56038

File <command-164724802993156>, line 13 11 catalogs = [r['catalog'] for r in spark.sql("SHOW CATALOGS").collect()] 12 if catalog not in catalogs: ---> 13 spark.sql(f"CREATE CATALOG IF NOT EXISTS {catalog}") 14 if catalog == 'dbdemos': 15 spark.sql(f"ALTER CATALOG {catalog} OWNER TO `account users`")


Is enabling unity catalog mandatory? If not, how do I proceed?

Thank you in advance 🙂

[Kaniz_Fatma]

Hi @Abhishek_blue, The error message you encountered, “[UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster,” indicates that the Unity Catalog feature is not enabled for your Databricks cluster.

Let’s address your questions:

1. Is enabling Unity Catalog mandatory?

   • The Unity Catalog is a powerful feature in Databricks that provides a unified metadata catalog for managing tables, views, and other data assets. It allows you to organize and query your data more efficiently.
   • However, enabling Unity Catalog is not mandatory. You can still work with Databricks without it, but you won’t have access to the additional features it provides.

2. How do you proceed?

   • If you want to enable Unity Catalog, follow these steps:
     • Workspace Configuration:
       • Go to your Databricks workspace.
       • Click on the “Workspace” tab.
       • Click “Update Workspace.”
       • Toggle the “Enable Unity Catalog” switch to the ON position.
       • Choose the corresponding metastore from the dropdown (the one assigned to your workspace).
       • Click “Update.”
     • Cluster Configuration:
       • Make sure you are using a Databricks Runtime (DBR) version greater than 11.
       • Check that credential passthrough is not enabled.
       • Verify that the Unity Catalog tag is associated with your cluster.
     • Notebook Usage:
       • In your notebook, use the USE CATALOG statement to work with the Unity Catalog. For example:

         %sql
         USE CATALOG my_new_catalog;
         USE SCHEMA my_schema;
         SELECT * FROM my_table;
         

   • If you encounter any issues during these steps, consider checking your Databricks account or subscription settings.

3. Cluster vs. Workspace:

   • The error message refers to the “cluster.” In this context:
     • The “workspace cluster” is the compute cluster associated with your Databricks workspace.
     • The “compute cluster” is where your notebooks run.
   • Ensure that Unity Catalog is enabled for both the workspace and the compute cluster.

Remember that Unity Catalog enhances data governance and simplifies data discovery, but it’s not strictly required. Feel free to proceed based on your specific needs! 😊

For more details, you can refer to the official Databricks documentation. If you encounter any further issues, don’t hesitate to ask for additional assistance! 🚀

Sources:

1. Databricks Community Forum: Unity Catalog Not Enabled
2. DevCodeF1: Databricks Community Version Error Message: UnityCatalog Not Enabled
3. Databricks Community Forum: Unable to Query Unity Catalog Tables from Notebooks
4. Stack Overflow: Unity Catalog Not Enabled on Cluster in Databricks
5. Databricks Community Forum: ‘Unity Catalog Volumes is not enabled on this instance’ Error¹²³⁴⁵