Issue with RAG - Review app - endpoint failed to stream the response
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ06-23-2025 07:32 AM
Hi Databricks Support,
I am facing issue for using - RAG review app endpoint url. Error message:
The endpoint failed to stream the response. Retrying without streaming.
The RAG endpoint url was created earlier and it worked with review app, at the time when we used databricks-meta-llama-3-1-70b-instruct. We paused the endpoint for a while and now we need to replace databricks-meta-llama-3-1-70b-instruct to databricks-meta-llama-3-3-70b-instruct. I tried deploying the new version of the model to replace this but the container creation is failing. Could you please in replacing llama3 in endpoint ?
Thanks,
Radhika
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ06-25-2025 05:06 AM
Can you share error details? I updated it without an porblme using mlflow.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ06-25-2025 09:08 AM
Below is the example that I am using, not sure if the libraries used here needs update for mlflow.
Similarly for libraries used in config as well may need update accordingly.
genai-cookbook/rag_app_sample_code/A_POC_app/pdf_uc_volume/03_deploy_poc_to_review_app.py at main ยท ...