MPT-30B

MPT-30B sets a new standard in the world of open-source foundation models, delivering enhanced performance and innovation. Developed using NVIDIA H100 Tensor Core GPUs, this transformational model boasts an impressive 8k context length, allowing for a deeper and more nuanced understanding of text. As part of the acclaimed MosaicML Foundation Series, MPT-30B offers open-source access and a license for commercial use, distinguishing itself as a highly accessible and powerful tool. It comes with specialized variants, including Instruct and Chat, suited for different applications. The model is optimized for efficient inference and training performance through technologies like ALiBi and FlashAttention, also featuring remarkable coding abilities thanks to its comprehensive pre-training data mixture. MPT-30B is strategically designed for single-GPU deployment, making it a convenient choice for a wide range of users.

Key Features

• Powerful 8k Context Length: Enhanced ability to understand and generate text with a longer context.

• NVIDIA H100 Tensor Core GPU Training: Leverages advanced GPUs for improved model training performance.

• Commercially Licensed and Open-Source: Accessible for both commercial use and community development.

• Optimized Inference and Training Technologies: Incorporates ALiBi and FlashAttention for efficient model usage.

• Strong Coding Capabilities: Pre-trained data mixture includes substantial code, enhancing programming proficiency.

FAQ

What is MPT-30B?

MPT-30B is a newly developed foundation model, part of the MosaicML Foundation Series, designed for advanced natural language understanding and generation.

On what hardware was MPT-30B trained?

It was trained on NVIDIA H100 Tensor Core GPUs which provide high computational power, important for handling the model's vast context length and complexity.

Are there any variants of the MPT-30B model?

In addition to the main MPT-30B model, there are two specialized variants named MPT-30B-Instruct and MPT-30B-Chat that excel in single-turn instruction following and multi-turn conversations respectively.

Is MPT-30B available for commercial use?

Yes, MPT-30B is licensed for commercial use under Apache License 2.0, making it open-source and suitable for use in commercial applications.

Can MPT-30B be deployed on a single GPU?

MPT-30B can be effectively deployed on a single GPU, specifically an NVIDIA A100-80GB in 16-bit precision or an NVIDIA A100-40GB in 8-bit precision.