BerriAI/litellm - GitHub
LiteLLM offers a universal solution for integrating various large language model (LLM) APIs into your applications by using a consistent OpenAI format. This tool allows seamless access to multiple pro
12,076
Votes
21,379
Views
4,484
Bookmarks
About
LiteLLM offers a universal solution for integrating various large language model (LLM) APIs into your applications by using a consistent OpenAI format. This tool allows seamless access to multiple providers such as Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, and Replicate, among others, without the need for adapting to each provider's specific API style. LiteLLM's features include input translation to different providers’ endpoints, consistent output formats, common exception mapping, and load balancing for high-volume requests. It supports over 100 LLM APIs, making it an indispensable tool for developers looking to leverage AI language models across different cloud platforms, all with the ease of OpenAI-style API calls.
Key Features
- Consistent Output Format: Guarantees consistent text responses across different providers.
- Exception Mapping: Common exceptions across providers mapped to OpenAI exception types.
- Load Balancing: Capable of routing over 1k requests/second across multiple deployments.
- Multiple Providers Support: Access to 100+ LLM providers using a single OpenAI format.
- High Efficiency: Translates inputs efficiently to provider's endpoints for completions and embeddings.
You may also like
More tools in Other
SuperU AI
A nocode tool to create voice AI agents for customer communications.
LLM Council
A tool to compare and synthesize multiple LLM responses.
@kuki_ai
Welcome to the world of Kuki, an award-winning artificial intelligence designed to bring entertainment to the digital age. Dive into engaging conversations with AI that's crafted to provide not just r
Integral Calculator - Wolfram|Alpha
The Integral Calculator provided by Wolfram|Alpha is a comprehensive tool designed for professionals, educators, students, and anyone with a need to solve complex mathematical integrals. By leveraging
AptlyStar.AI
A tool to create and manage AI bots for businesses.
PureCode.ai
A tool to automate coding tasks through codebase-aware code generation.
AI Dungeon
AI Dungeon is a text-based adventure game where you lead the story and the AI creates the world around you. It offers endless possibilities by generating unique characters, settings, and scenarios bas
G3D.AI {Jedi}
G3D.AI {Jedi} is a generative AI tool for game creation that enables game creators to build beautiful and novel games in a fraction of the time. With a suite of tools designed to supercharge creativit
Wan 2.7 AI Video Generator
Wan 2.7 AI Video Generator transforms still images into high-quality, realistic 1080P videos with dynamic motion and advanced controls. It targets creators, marketers, e-commerce professionals, and di
Verbacall
A platform that automatically answers, qualifies, and follows up on calls 24/7.
PrompTessor
A tool that optimizes text for clarity, tone, and grammar without requiring prompt engineering skills.
PlayMix AI
A tool to create playable games from ideas.