Error when querying RAG serving endpoint

[mark_goodwin]

I built a RAG by cloning the databricks example (create a vector search index of databricks documentation and then a RAG).

When I try and call the RAG via the serving endpoint from a notebook, I get the following error:

dictionary update sequence element #0 has length 309; 2 is required

I can't figure out what I've done wrong.  When I use the Serving interface in databricks and ask it a question, it works fine.  Here's my code:

chat_model = ChatDatabricks(
    endpoint="my_rag",
    temperature=0.1,
    max_tokens=256,

messages = [
    ("user", "What questions can I ask?"),
]
chat_model.invoke(messages)

Any tips or hints on what my problem is I would be grateful!  Thanks.

[Shua42]

Hi @mark_goodwin ,

I believe this has to do with the formatting of the messages. In the serving UI, the formatting is abstracted out for you, but you may need to format it as a list of dictionaries for the invoke() call, such as:

messages = [
    {"role": "user", "content": "What questions can I ask?"}
]

Try that out and let me know if it yields any different results.

[mark_goodwin]

Hello @Shua42 thanks for getting back to me!

It seems that for ChatDatabricks, it does not expect a dictionary, it's expecting an array of tuples, so doing the above gave me an error.

When I use your message format in connecting via API, that works.  But it's just for this ChatDatabricks where I am struggling.