Retrieved Docs in Message History

[gwfj]

Hi,

I've deployed a Langchain model via MLFlow log model and a deployment agent (following this https://notebooks.databricks.com/demos/llm-rag-chatbot/index.html#)

How do I add additional content to the message history for example, I'd like to add retrieved docs to the chat_history so that the LLM is aware of past retrievals. 

I've had a look at Langchain's doc on adding chat history (https://python.langchain.com/v0.2/docs/how_to/qa_chat_history_how_to/) but I am not too sure whether this will work with MLFlow. 

Do you have any tips/documentation I can refer to? Any assistance is appreciated. Thanks!

Gary

[davidhuang]

In the notebook `02-Advanced-Chatbot-Chain`, from the RAG demo notebooks, shows how to incorporate chat history.

[Brian-Nowak]

Hi @davidhuang thanks for pointing to that demo! That implementation of chat history did work in my experience, but it's a little simplistic and could be token-intensive if the history needs to be maintained as a list like that. Has Databricks done any demos / POCs on using LangGraph for stateful management of chat history? Reference to the tutorial here 

I am trying to implement this but the blocker I'm currently facing is I can't figure out how to pass a thread_id or other config data into the graph from the Model Serving Endpoint. Any chance you've tackled this? Thanks!