Trouble Using Foundation Models API with Instructor

[Gromduraz]

Hello,

I'm attempting to use the instructor library for validating the output of a foundation model (DBRX in this case). Here's my code:

import instructor
from instructor.client import Instructor
from pydantic import BaseModel
from openai import OpenAI

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(
    OpenAI(
        api_key=mykey,
        base_url=myendpoint,
    )
)


chat_completion = client.chat.completions.create(
        messages=[
            {
                "role": "system",
                "content": "You are an AI assistant",
            },
            {"role": "user", "content": "Create a user"},
        ],
        model="databricks-dbrx-instruct",
        response_model=User,
        max_tokens=4000,
        temperature=0.8,
        tool_choice="auto",
    )

print(chat_completion.choices[0].message.content)

When I run this I get:

BadRequestError: Error code: 400 - {'error_code': 'BAD_REQUEST', 'message': 'Bad request: json: unknown field "tool_choice"\n'}

I assume the conflict here is that the instructor library was designed to send it's requests directly to OpenAI's servers, while we're using the Databricks Foundation Models API and it's passing along the "tool_choice" field which the endpoint isn't expecting? 

Am I missing something obvious? Can I get this to run as-is or would this require a change to the functionality of the Foundation Models API to run? Are there alternatives to Instructor that I should try? Thanks!

[Gromduraz]

Reviewing my posted code I should mention that the 'tool_choice' argument I'm passing is a default argument of the OpenAI client object. The code still breaks in the same way if I remove that argument from the function call. I think this is out of my hands to fix as it would require either a change to what instructor sends during the payload to DBX or how DBX interprets the payload. 

I did however find a workaround using LangChain's PydanticValidator. That seemed to work although it's not as direct of an implementation as instructor.