Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling

kirti-11 — Wed, 19 Mar 2025 16:29:34 GMT

Dear Databricks Community,

I am seeking assistance with an issue I've encountered while deploying a model on Databricks. When invoking the serving endpoint, I receive the following error message sometimes:

400 Client Error: Error: Model response did not respect the required format. Please consider retrying or using a more straightforward prompt.

Context:

Endpoint URL: https://adb-*****************.11.azuredatabricks.net/serving-endpoints/databricks-meta-llama-3-3-70b-instruct/invocations
Model: databricks-meta-llama-3-3-70b-instruct
Steps Taken So Far:
1. Reviewed Input Prompts: Ensured that the input prompts are straightforward and free from complexities that might confuse the model.
2. Checked Model Response Structure: Verified that the model's output aligns with the expected format required by the serving endpoint.
Observations:
- Intermittent Occurrence: The error does not occur consistently; some requests are processed successfully, while others result in the aforementioned error.
Additional Context:
- Similar Issues: I have come across similar issues reported by other users, such as the one discussed in this GitHub issue, where the error message 'Model response did not respect the required format' was encountered.
Despite these efforts, the issue persists intermittently. I would greatly appreciate any insights or recommendations from the community to resolve this error.

Re: Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling

jericksoncea — Fri, 16 May 2025 14:22:01 GMT

Same issue here... developing a langraph tool-calling agent, and for certain (but not all) questions, I get the same error.

Any luck resolving?

Re: Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling

jericksoncea — Mon, 19 May 2025 10:21:37 GMT

FYI: I changed to databricks-claude-3-7-sonnet and no longer have the issue.

topic Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling in Generative AI

Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling

Re: Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling

Re: Databricks Serving Endpoint 400 Error: Model Response Format Issue while langgraph tool calling