topic Re: LLM with the largest context window in Generative AI

LLM with the largest context window

royinblr11 — Tue, 06 May 2025 09:42:18 GMT

A Generative AI Engineer is tasked with developing an application that is based on an open-source large language model (LLM). They need a foundation LLM with a large context window. Which model fits this need?

DBRX,
Llama2-70B,
DistilBert
MPT-30B.

DBRX has a larger context window compared to MPT-30B. DBRX has a 32K token context window, while MPT-30B has an 8k token context window but the Answer has mentioned as MPT-30B. Can anyone please help here? Thanks in advance.

Re: LLM with the largest context window

sarahbhord — Wed, 07 May 2025 15:44:32 GMT

Hey royinblr11,

Where did this question come from and when was it published? You are correct in that the latest DBRX model has a 32k token context window, larger than MPT-30B's 8k context window. Our latest publication on this stat was March 2024. If the question was published before then, it might be out of date.

https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm

Re: LLM with the largest context window

lingareddy_Alva — Wed, 07 May 2025 20:38:17 GMT

@royinblr11

You're absolutely right to question the answer — the correct model for an application needing a foundation LLM with a large context window is: DBRX

Why DBRX is the Best Fit:

It is a foundation model, designed for generation tasks.
It supports a 32K token context window out of the box — ideal for handling long documents, chats, or code.
It is open-source and production-ready for enterprise-level generative AI tasks.

So, if the goal is:

Build a generative application using an open-source model with a large context window,

Then the best answer is DBRX, not MPT-30B.