Databricks' dolly-v2-12b

Databricks introduces dolly-v2-12b, an inventive language model providing high-quality, instruction-following capabilities. This 12-billion-parameter model is built on Pythia-12b, yielding exceptional performance, not inherent in its foundational model. Dolly-v2-12b is optimized for commercial use, having been fine-tuned on a diverse mix of instructions covering areas like brainstorming, classification, and summarization. While it may not be at the forefront of AI models, its proficiency in instruction adherence is remarkable. Available in varying sizes, dolly-v2-12b is complemented by the smaller dolly-v2-7b and dolly-v2-3b models. Users can leverage this model easily with Transformers and PyTorch using the Hugging Face platform, tapping into the practical insights and extensive dataset, databricks-dolly-15k, to power their AI applications.

Key Features

• Large Model Size: Boasts a massive 12 billion parameters for intricate text generation.

• Advanced Instruction Following: Exhibits high-quality instruction following, surpassing its foundational model.

• Fine-Tuned Dataset: Utilizes the databricks-dolly-15k dataset, comprised of diverse instructive interactions.

• Open Science Commitment: Released under a permissive license, dolly-v2-12b furthers the democratization of AI.

• Flexible Usage: Supports various GPU configurations and can be seamlessly integrated with Transformers library.

FAQ

What is dolly-v2-12b?

Dolly-v2-12b is a 12 billion parameter language model derived from EleutherAI's Pythia-12b and instruction-tuned on ~15k records from Databricks.

How do I use dolly-v2-12b?

You can run the model using the transformers library on a machine with GPUs. The model repository provides a custom InstructionTextGenerationPipeline for this purpose.

Does dolly-v2-12b have any limitations?

Dolly-v2-12b is not considered state-of-the-art and exhibits certain limitations like difficulty with complex prompts, mathematical operations, and open-ended question answering.

Where can I find the dolly-v2-12b model?

The dolly-v2-12b model is available on Hugging Face and can be found under the owner Databricks, Inc.

Are there any smaller versions of dolly-v2-12b?

Yes, there are smaller variants like dolly-v2-7b with 6.9 billion parameters and dolly-v2-3b with 2.8 billion parameters.