Pydantic usage for structured output with provisioned LLM

[fcardoze]

Hi,

I am looking for a resource that has examples of using Pydantic with a provisioned LLM on Databricks to get structured output.

I can find many examples of using Pydantic with LLMs, but not on the Databricks.

My use case is to extract text from images using one of the provisioned LLMs on Databricks into a structured format. For the LLM I would like to use Claude. Any help is greatly appreciated.

Regards,

Frank

[jAAmes_bentley]

Are you looking to use models on batch or in a more traditional framework like LangChain? If the latter, you could use the Pydantic Output Parser with databricks-langchain?:

databricks-langchain · PyPI
PydanticOutputParser — 🦜🔗 LangChain documentation

[fcardoze]

Thanks for the reply. LangChain is a framework that relies on Pydantic per my understanding. I was trying to keep the number of frameworks to a minimum for my use case which is strictly to get a structured output. I am still learning so perhaps I am approaching this the wrong way? 

[jAAmes_bentley]

I think it depends what your overall usecase is.

• If you're looking to extract text from images / documents specifically using Databricks then you could consider ai_parse which provides a structured extraction of text and OCR content from files: ai_parse_document function | Databricks Documentation
• If you're looking to query an LLM in bulk / batch, you should consider calling Claude with ai_query, which supports structured outputs to a certain degree using the responseFormat argument: Databricks Documentation
• If you're looking to ping an LLM endpoint in a more one-at-a-time way, then you'll need to query the Databricks endpoints somehow.
  • The often touted "simplest" way is to use the openai library, and they do have support for structured outputs using tools such as pydantic: Introducing Structured Outputs in the API | OpenAI
  • IMO, I find LangChain a lot easier to work with, but that might just be because I've been using it for like two years or so I've just learned to think about LLMs in a chain-y way, and as I said above it has a Pydantic output parser too: PydanticOutputParser — 🦜🔗 LangChain documentation

In the end, I wouldn't say LangChain is an unecasserily heavy framework, and it carries a lot of tools, docs, and examples which can help you upskill quickly. If you really want to keep it as minimal as possible, then use the openai library. However, as said, I'd personally recommend the LangChain links I've given above.

[fcardoze]

Thank you!