- 477 Views
- 1 replies
- 0 kudos
How to utilize clustered gpu for large hf models
Hi,I am using clustered GPU(driver -1GPU and Worker-3GPU), and caching model data into unity catalog but while loading model checkpoint shards its always use driver memory and failed due insufficient memory.How to use complete cluster GPU while loadi...
- 477 Views
- 1 replies
- 0 kudos
- 0 kudos
1. Are you using any of the model parallel library, such as FSDP or DeepSpeed? Otherwise, every GPU will load the entire model weights. 2. If yes in 1, Unity Catalog Volumes are exposed on every node at /Volumes/<catalog>/<schema>/<volume>/..., so w...
- 0 kudos
- 5284 Views
- 10 replies
- 7 kudos
Resolved! Insufficient Permission Error When Serving RAG Model with Multiple Vector Search Indexes
Hi Community,I’m currently working on a Retrieval-Augmented Generation (RAG) use case in Databricks. I’ve successfully implemented and served a model that uses a single Vector Search index, and everything works as expected.However, when I try to serv...
- 5284 Views
- 10 replies
- 7 kudos
- 6086 Views
- 4 replies
- 0 kudos
Resolved! LangGraph MemorySaver checkpointer usage with MLflow
Hi everyone.I am working on a graph that utilizes the MemorySaver class to incorporate short-term memory. This will enable me to maintain a multi-turn conversation with the user by storing the chat history.I am using the MLflow "models from code" fea...
- 6086 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @moemedina. No, I didn't.I'm considering using ChatModel/ChatAgent class to wrap the graph and be able to move on. However, the MLflow documentation is still referring to ChatModel where Chat Agent is the latest recommendation:MLflow ChatModel Doc...
- 0 kudos