topic Re: Retrieved Docs in Message History in Generative AI

Retrieved Docs in Message History

gwfj — Sun, 01 Sep 2024 21:48:20 GMT

Hi,

I've deployed a Langchain model via MLFlow log model and a deployment agent (following this https://notebooks.databricks.com/demos/llm-rag-chatbot/index.html#)

How do I add additional content to the message history for example, I'd like to add retrieved docs to the chat_history so that the LLM is aware of past retrievals.

I've had a look at Langchain's doc on adding chat history (https://python.langchain.com/v0.2/docs/how_to/qa_chat_history_how_to/) but I am not too sure whether this will work with MLFlow.

Do you have any tips/documentation I can refer to? Any assistance is appreciated. Thanks!

Gary

Re: Retrieved Docs in Message History

davidhuang — Thu, 24 Oct 2024 06:01:11 GMT

In the notebook `02-Advanced-Chatbot-Chain`, from the RAG demo notebooks, shows how to incorporate chat history.

Re: Retrieved Docs in Message History

Brian-Nowak — Tue, 12 Nov 2024 19:28:55 GMT

Hi @davidhuang thanks for pointing to that demo! That implementation of chat history did work in my experience, but it's a little simplistic and could be token-intensive if the history needs to be maintained as a list like that. Has Databricks done any demos / POCs on using LangGraph for stateful management of chat history? Reference to the tutorial here

I am trying to implement this but the blocker I'm currently facing is I can't figure out how to pass a thread_id or other config data into the graph from the Model Serving Endpoint. Any chance you've tackled this? Thanks!