topic How to use CrewAI with Databricks-hosted LLMs (not OpenAI) ? in Generative AI

How to use CrewAI with Databricks-hosted LLMs (not OpenAI) ?

lucami — Tue, 20 Jan 2026 16:25:27 GMT

I am trying to use CrewAI to train an agent with a Databricks-hosted LLM (such as Llama 2 instruct or DBRX) via the databricks_langchain integration. I want CrewAI to use only my Databricks LLM and not requiring OpenAI authentication.

However, when I use ChatDatabricks to both the Agent and the Crew, CrewAI still tries to use OpenAI and fails with an authentication error. If I try to use custom llm, I get litellm.AuthenticationError: databricksException.

What is the correct way to configure CrewAI to use only a Databricks-hosted LLM ?
Should I pass the LLM only to the Agent, or also to the Crew?
Are there any best practices or working examples for this setup in Databricks notebooks?

Thank you!

Re: How to use CrewAI with Databricks-hosted LLMs (not OpenAI) ?

lingareddy_Alva — Thu, 22 Jan 2026 17:24:01 GMT

@lucami

CrewAI defaults to OpenAI unless you explicitly override it.
You must pass your Databricks LLM to both the Agent and the Crew.

Create the Databricks LLM:
Use it in the Agent and in the Crew (this is critical)

Re: How to use CrewAI with Databricks-hosted LLMs (not OpenAI) ?

pavannaidu — Wed, 04 Feb 2026 17:11:18 GMT

@lucami

To configure CrewAI to use only Databricks-hosted models without OpenAI dependencies

Use CrewAI's LLM class (which wraps litellm) rather than ChatDatabricks
Pass the LLM at both levels: agents use it to complete their tasks, and the crew uses it to coordinate and orchestrate the overall workflow (as @lingareddy_Alva already suggested)
Enable MLflow tracing for comprehensive execution tracking -- this is super helpful to trace agent execution and model calls.

CrewAI is supported in MLflow! I tried to replicate the example in MLflow page to use Databricks hosted models > https://gist.github.com/pavannaidu/a61c25fe76b07601bee4aa5138aefffe and see the attached trace for model calls -- hope this helps!