Galactica

Galactica is an advanced large language model specifically designed to handle the complexities of scientific knowledge. Its ability to store, combine, and reason about vast amounts of scientific information makes it an invaluable resource to overcome the challenges of information overload in the science community. Developed by a team led by Ross Taylor, Marcin Kardas, and several other contributors, Galactica stands out with its superior performance on a variety of scientific tasks. By outperforming existing models such as GPT-3 on technical knowledge probes and Chinchilla on mathematical MMLU, Galactica has shown potential to serve as a potent new interface for scientific exploration and discovery. Notably, Galactica has achieved state-of-the-art results on downstream tasks such as PubMedQA and MedMCQA dev, substantiating its value addition to the scientific community. The model is open-sourced, encouraging broad usage and contribution among researchers and institutions globally.

Key Features

• Superior Performance: Outperforms GPT-3, Chinchilla, and PaLM 540B on scientific tasks.

• Advanced Reasoning: Demonstrates high proficiency in mathematical and technical knowledge reasoning.

• State-of-the-Art Results: Achieves new benchmarks on PubMedQA and MedMCQA dev tasks.

• Open Source: Available to the scientific community for wider adoption and contribution.

• Diverse Training Corpus: Trained on a vast scientific corpus including papers, reference material, and knowledge bases.

FAQ

What is Galactica?

Galactica is a large language model designed to organize and reason about scientific knowledge, aimed at aiding scientific research and discovery.

Has Galactica outperformed other existing language models?

Yes, Galactica has outperformed other models like GPT-3 on technical knowledge probes, Chinchilla on mathematical MMLU, and PaLM 540B on MATH.

What type of data was Galactica trained on?

Galactica has been trained on a large scientific corpus, including papers, reference materials, knowledge bases, and other scientific sources.

Is Galactica open source, and why?

The developers of Galactica have made the model open source to contribute to the advancement of scientific research in the community.

What are some downstream tasks where Galactica has shown promising results?

Galactica has a demonstrated capacity to outperform on downstream tasks like PubMedQA, achieving 77.6%, and MedMCQA dev, achieving 52.9% in benchmarks.