Dbdemo: LLM Chatbot With Retrieval Augmented Generation (RAG)

[cmunteanu]

Hello All,

I am trying to follow the dbdemo called 'llm-rag-chatbot' available at the following link. The setup works Ok, and I have imported from the Databricks Marketplace an embedding model that is:

• bge_large_en_v1_5

Running the notebook called: 01-Data-Preparation-and-Index  I am stuck with an error when trying to create a Vector Search Index with Managed Embeddings and the BGE model that I have setup as a serving endpoint, previously. More specifically, the Vector Search endpoint provisions succesfully, but when executing the index creation and syncronization method: create_delta_sync_index, I get the following error:

----

Exception: Response content b'{"error_code":"INVALID_PARAMETER_VALUE","message":"Model serving endpoint bge-large-en configured with improper input: {\\"error_code\\": \\"BAD_REQUEST\\", \\"message\\": \\"Failed to enforce schema of data \' 0\\\\n0 Welcome to databricks vector search\' with schema \'[\'input\': string (required)]\'. Error: Model is missing inputs [\'input\']. Note that there were extra inputs: [0]\\"}"}', status_code 400

----

My code that calls this method is:

if not index_exists(vsc, VECTOR_SEARCH_ENDPOINT_NAME, vs_index_fullname):

  print(f"Creating index {vs_index_fullname} on endpoint {VECTOR_SEARCH_ENDPOINT_NAME}...")

  vsc.create_delta_sync_index(

    endpoint_name=VECTOR_SEARCH_ENDPOINT_NAME,

    index_name=vs_index_fullname,

    source_table_name=source_table_fullname,

    pipeline_type="TRIGGERED",

    primary_key="id",

    embedding_source_column='content', #The column containing our text

    embedding_model_endpoint_name='bge-large-en'

    #embedding_model_endpoint_name='gte_large' 

  )

I have tried changing to a different embedding model (GTE_LARGE), but still getting the above error.

I guess there is a incompatibilty between the input schema of the embedding model and the schema expected by the vector search endpoint.

Has any of you encountered this problem?  I would appreciate if you could give me a hint on how to solve it using an embedded model from Databricks Marketplace.

Thanks ! 

[Kaniz · Accepted solution]

Hi @cmunteanu, 

• Ensure that the input schema of your embedding model aligns with what the Vector Search endpoint expects. The ‘input’ column is crucial for the model to process text data correctly.
• Verify the input requirements of the bge_large_en_v1_5 model. It should expect a column named ‘input’ containing text data.
• Confirm that your data pipeline provides the necessary input format to the model.
• When creating the Vector Search Index, ensure that you specify the correct parameters:
  • embedding_source_column: This should match the column name containing your text data (e.g., ‘content’).
  • embedding_model_endpoint_name: Use ‘bge-large-en’ as you’ve set up this model as a serving endpoint.
• If you’ve made changes to the model or schema, consider reindexing the Vector Search.
• Note that the LLM (Language Model) cannot be modified after indexing, so any changes require reindex...¹
• If you continue to face issues, consider using multimodal embeddings (such as CLIP) that can handle ...².
• Retrieve using similarity search and link to images in a docstore.

[cmunteanu]

Hello @Kaniz , thanks a lot for the information you provided. Anyhow, I have managed a workaround, by pre-computing the embeddings for each chunk.  I have created an embedding column on the source table and used this column as input to the create_delta_sync_index method.

That is: substitute parameter  embedding_source_column='content' for:

embedding_dimension=1024,

embedding_vector_column="embedding"

and the syncronization of the index with the source table worked just fine.