OpenHermes-13B

Discover OpenHermes-13B, an advanced fine-tuned model from teknium that leverages the robust GPT-4 generated dataset collected from diverse AI solutions. Meticulously trained on a fully open-source dataset comprising 242,000 entries, OpenHermes-13B was crafted using contributions from industry leaders including Teknium, the WizardLM Team, and Microsoft to name a few. This model aims to enhance text generation via open datasets, specifically engineered to remove AI disclaimers and refusals to streamline communication outputs. Its training process, facilitated by a16z's sponsorship and main_horse's compute resources, embraces transparency by allowing public access to the WANDB Project. Dive into the model's impressive benchmark results, its average score comparison with similar models, and explore the particular hyperparameters that have shaped its training.

Key Features

• Advanced Training Dataset: Leveraged the Hermes dataset with fine-tuning on GPT-4 generated data across the AI landscape.

• Open Source Contribution: Emphasized an open-source approach with datasets from multiple AI industry contributors.

• Strategic Filtering: Removed specific content such as OpenAI refusals and disclaimers to improve output quality.

• Benchmark Performance Analysis: Showcased benchmark results across different suites like GPT4ALL, BigBench, and AGI-Eval.

• Transparent Training Procedure: Detailed the training process with publicly available WANDB Project logs.

FAQ

What is OpenHermes-13B?

OpenHermes-13B is a highly advanced AI model fine-tuned on datasets primarily generated by GPT-4, developed by teknium and available via Hugging Face.

What datasets were used in training OpenHermes-13B?

The model was trained on dataset contributions from GPTeacher, Airoboros, Camel-AI, CodeAlpaca, WizardLM, and Microsoft's GPT4-LLM and Unnatural Instructions datasets.

Is the WANDB Project for OpenHermes-13B public?

Yes, the WANDB Project is publicly available and can be examined to understand the training and development process of OpenHermes-13B.

Who sponsored the development of OpenHermes-13B?

The development of OpenHermes-13B was sponsored by a16z and supported with compute access by main_horse.

What improvements does OpenHermes-13B hold over previous models?

OpenHermes-13B has shown a slight improvement in benchmarks like GPT4ALL Suite and BigBench Suite, with some degradation in AGIEval compared to the original Hermes model.