topic Re: Error when querying RAG serving endpoint in Generative AI

Error when querying RAG serving endpoint

mark_goodwin — Tue, 27 May 2025 14:55:24 GMT

I built a RAG by cloning the databricks example (create a vector search index of databricks documentation and then a RAG).

When I try and call the RAG via the serving endpoint from a notebook, I get the following error:

dictionary update sequence element #0 has length 309; 2 is required

I can't figure out what I've done wrong. When I use the Serving interface in databricks and ask it a question, it works fine. Here's my code:

chat_model = ChatDatabricks( endpoint="my_rag", temperature=0.1, max_tokens=256, messages = [ ("user", "What questions can I ask?"), ] chat_model.invoke(messages)

Any tips or hints on what my problem is I would be grateful! Thanks.

Re: Error when querying RAG serving endpoint

Shua42 — Tue, 27 May 2025 15:19:06 GMT

Hi @mark_goodwin ,

I believe this has to do with the formatting of the messages. In the serving UI, the formatting is abstracted out for you, but you may need to format it as a list of dictionaries for the invoke() call, such as:

messages = [ {"role": "user", "content": "What questions can I ask?"} ]

Try that out and let me know if it yields any different results.

Re: Error when querying RAG serving endpoint

mark_goodwin — Tue, 03 Jun 2025 17:13:15 GMT

Hello @Shua42 thanks for getting back to me!

It seems that for ChatDatabricks, it does not expect a dictionary, it's expecting an array of tuples, so doing the above gave me an error.

When I use your message format in connecting via API, that works. But it's just for this ChatDatabricks where I am struggling.