Trouble Using Foundation Models API with Instructor

Gromduraz
New Contributor II

Hello,

I'm attempting to use the instructor library for validating the output of a foundation model (DBRX in this case). Here's my code:

 

import instructor
from instructor.client import Instructor
from pydantic import BaseModel
from openai import OpenAI

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(
    OpenAI(
        api_key=mykey,
        base_url=myendpoint,
    )
)


chat_completion = client.chat.completions.create(
        messages=[
            {
                "role": "system",
                "content": "You are an AI assistant",
            },
            {"role": "user", "content": "Create a user"},
        ],
        model="databricks-dbrx-instruct",
        response_model=User,
        max_tokens=4000,
        temperature=0.8,
        tool_choice="auto",
    )

print(chat_completion.choices[0].message.content)

 

When I run this I get:

 

BadRequestError: Error code: 400 - {'error_code': 'BAD_REQUEST', 'message': 'Bad request: json: unknown field "tool_choice"\n'}

 

I assume the conflict here is that the instructor library was designed to send it's requests directly to OpenAI's servers, while we're using the Databricks Foundation Models API and it's passing along the "tool_choice" field which the endpoint isn't expecting? 

Am I missing something obvious? Can I get this to run as-is or would this require a change to the functionality of the Foundation Models API to run? Are there alternatives to Instructor that I should try? Thanks!