Tool Calls with Workspace Models
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2025 07:02 AM
I recently followed the blog post on running deepseek llama distilled. I then served it via Serving Endpoints with provisioned throughput. In my use case I am using pydantic-ai to build out some simple agents for testing. It seems with this style of deployment I'm unable to have the agent make multiple tool calls. Once the llm responds with an 'assistant' role if I pass the full message history back in with the response from that tool call then I get the following error:
Model does not support continuing the chat past the first tool call
I believe this has to do with the way the serving endpoints are being configured when using 'llm/v1/chat' but I could be wrong.
Is a way around this to build out the inference configuration manually? Will I lose any functionality?
The only models this currently works on is the foundational models that support Function calling. ex: databricks-meta-llama-3-3-70b-instruct.
Any guidance here would be great!