Hi @Saurabh2406 ,

This sounds like a fairly advanced use case - are you in touch with your account team at Databricks? They would be able to provide you with more detailed guidance on this use case. They could also get you connected with internal specialists.

In the meantime, you should take a look at these resources to help:

• Tracing LangGraph
• Vector Search Retrieval Quality Guide
• Consider enabling AI Gateway for your external model endpoints to enable traffic policies, payload logging, and rate limiting
  • Other general recommendations:
    • Prefer streaming responses for chat/completions to shave tail latency
    • Batch inference: for large volumes, use SQL/Python ai_query when invoking Databricks‑hosted models (including Gemini 2.5 Pro/Flash) to process data at scale with automatic backend capacity management
• Use MLFlow trace timelines to find slow spans