Building DBRX-class Custom LLMs with Mosaic AI Training

Sujitha — Fri, 17 May 2024 13:03:24 GMT

We recently introduced DBRX: an open, state-of-the-art, general-purpose LLM. DBRX was trained, fine-tuned, and evaluated using Mosaic AI Training, scaling training to 3072 NVIDIA H100s and processing more than 12 trillion tokens in the process.

Training LLMs, and in particular MoE models such as DBRX, is hard. It requires overcoming many infrastructure, performance, and scientific challenges. Mosaic AI Training was purposely built to address these challenges and was battle-tested through the training of DBRX, the MPT series of models, and many other LLMs such as Ola’s Krutrim, AI2’s OLMo, Dynamo AI’s Dynamo 8B, Refuel’s LLM-2, and others.

Continue to read more here.

Re: Building DBRX-class Custom LLMs with Mosaic AI Training

Insurance — Thu, 13 Jun 2024 17:51:26 GMT

exciting!

topic Building DBRX-class Custom LLMs with Mosaic AI Training in Announcements

Building DBRX-class Custom LLMs with Mosaic AI Training

Re: Building DBRX-class Custom LLMs with Mosaic AI Training