A recent supply chain attack compromised versions 1.82.7 and 1.82.8 of LiteLLM, a popular framework for interacting with large language models (LLMs), via the PyPI repository. This incident has prompted the community to seek valid and secure alternatives.

Open Source Alternatives to LiteLLM

Among the most promising open-source alternatives are:

  • Bifrost: Written in Go, Bifrost is proposed as a direct replacement for LiteLLM, boasting up to 50x lower P99 latency. It supports over 20 providers and is released under the Apache 2.0 license. Migration from LiteLLM requires a simple base URL change.
  • Kosong: Developed by Kimi and used in the Kimi CLI, Kosong is an agent-oriented LLM abstraction layer. It unifies message structures and asynchronous tool orchestration with pluggable chat providers. It supports OpenAI, Anthropic, Google Vertex, and other API formats.
  • Helicone: An AI gateway with strong analytics and debugging capabilities. It supports over 100 providers. Compared to the first two alternatives, Helicone is heavier but offers richer features on the observability front.