cancel
Showing results for 
Search instead for 
Did you mean: 
Generative AI
Explore discussions on generative artificial intelligence techniques and applications within the Databricks Community. Share ideas, challenges, and breakthroughs in this cutting-edge field.
cancel
Showing results for 
Search instead for 
Did you mean: 

Building RAG without Catalog

Aminsn
New Contributor II

All the tutorials I have seen about building a RAG within databricks involves using the Catalog option. For some reason my organization doesn't support that option and I can't use the unity catalog to build a RAG. Can I still use other features such as vector search and all the easy to use features that databricks provides without using the catalog? Can someone please refer me to a tutorial of building a RAG with databricks without using the catalog option? 

1 ACCEPTED SOLUTION

Accepted Solutions

raphaelblg
Databricks Employee
Databricks Employee

Hi @Aminsn

Having Unity Catalog enabled is a requirement for vector search. Source: Vector Search - Requirements. If you're referencing this tutorial Creating High Quality RAG Applications with Databricks I think that many of the required features also rely on UC and there's not much you can do to avoid that. I'm not aware of a non-UC tutorial for the scenario you described.

These are the docs on how to enable a workspace for UC, in case you'd like to proceed in this way: https://docs.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks

View solution in original post

3 REPLIES 3

raphaelblg
Databricks Employee
Databricks Employee

Hi @Aminsn

Having Unity Catalog enabled is a requirement for vector search. Source: Vector Search - Requirements. If you're referencing this tutorial Creating High Quality RAG Applications with Databricks I think that many of the required features also rely on UC and there's not much you can do to avoid that. I'm not aware of a non-UC tutorial for the scenario you described.

These are the docs on how to enable a workspace for UC, in case you'd like to proceed in this way: https://docs.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks

Aminsn
New Contributor II

Thank you @raphaelblg , I asked our data engineer, who is our databricks admin, to enable UC and this was his response: 

"It's fairly complex because the Unity Catalog aims to have a single metastore used across all workspaces. We want to implement it at some point due to its numerous benefits. However, doing it correctly requires a significant amount of time."

My data engineering knowledge is very limited, but judging by the instructions provided in the link you shared with me, I don't see any complexity in enabling it. Is there something I am missing regarding the complexities of enabling that option on databricks? 

raphaelblg
Databricks Employee
Databricks Employee

Hi @Aminsn

You don't need to enable all workspaces to Unity Catalog at once, it's possible to enable them by demand. 

We have a very straightforward guide on how to enable UC. The first step is to Create a Unity Catalog metastore and then Enable your workspace for Unity Catalog. It requires some degree of product knowledge but we have docs detailing every step.

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group