LiteLLM
| Developer(s) | BerriAI |
|---|---|
| Initial release | 2023 |
| Repository | https://github.com/BerriAI/litellm |
| Written in | Python and TypeScript |
| Engine | |
| Type | LLM gateway |
| License | MIT License |
| Website | www |
Search LiteLLM on Amazon.
LiteLLM is an open-source large language model (LLM) gateway developed by BerriAI. Founded by Ishaan Jaffer and Krrish Dholakia[1]. It provides a unified OpenAI-compatible application programming interface (API) for accessing and routing requests to different model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library within applications.
Functionality
LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI-compatible format, allowing applications to switch between models without separately implementing each provider's interface.[2][3]
The software also provides model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.[3]
LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.[4]
Security
In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.[5]
Compatibility
Supported models
LiteLLM supports more than 100 large language models through a common OpenAI-compatible interface. Notable supported model families include:[3]
Compatible AI coding software
LiteLLM can be used as an inference gateway for AI-assisted software development tools, routing their model requests through the LiteLLM proxy. Documented integrations include:[6]
See also
- List of AI-assisted software development tools
- OpenRouter – proprietary LLM gateway
References
- ↑ "LiteLLM: Call every LLM API like it's OpenAI [100+ LLMs]". Y Combinator. Retrieved 2026-10-11.
- ↑ MSV, Janakiram (May 15, 2025). "LiteLLM: An open-source gateway for unified LLM access". InfoWorld. Retrieved September 9, 2026.
- ↑ 3.0 3.1 3.2 "LiteLLM - Getting Started". LiteLLM. BerriAI. Retrieved September 9, 2026.
- ↑ "Introducing the Model Context Protocol". Anthropic. November 25, 2024. Retrieved September 9, 2026.
- ↑ Singh, Jagmeet (March 31, 2026). "Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project". TechCrunch. Retrieved September 9, 2026.
- ↑ "LiteLLM Proxy CLI". GitHub. BerriAI. Retrieved September 10, 2026.
- ↑ "Configure LiteLLM Provider". Cline. Retrieved September 10, 2026.
External links
- Official website
- LiteLLM on GitHub
Template:Large language models Template:AI-assisted software development
This article "LiteLLM" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:LiteLLM. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
