AI Gateways & LLM Routing
AI gateways sit between your app and model providers, giving you a single API to many models plus caching, retries and fallbacks, rate limiting, budgets, logging, and key management. They reduce provider lock-in and add reliability and cost controls to LLM traffic.
| Cloudflare AI Gateway Proxy your AI traffic through Cloudflare for caching and analytics. | Proprietary | Managed | Free tier + paid | Major providers | Yes | Yes | Yes |
| Helicone Open-source LLM logging and monitoring via a proxy. | Open source (Apache-2.0) | Managed, Self-hosted | Free tier + paid | — | — | — | — |
| Kong AI Gateway AI features on the Kong API gateway. | Open source (Apache-2.0 (Kong Gateway); enterprise features separate) | Self-hosted, Managed | Open source (free) | Major providers | Yes | Yes | Yes |
| LiteLLM Open-source proxy and SDK for 100+ LLM providers. | Open source (MIT) | Self-hosted, Managed | Open source (free) | 100+ providers | Yes | Yes | Yes |
| OpenRouter One API and account for hundreds of models. | Proprietary | Managed | Usage-based | 300+ models | No | Yes | Yes |
| Portkey AI gateway with observability, guardrails, and prompt management. | Open source (Apache-2.0 (gateway core); platform is managed) | Managed, Self-hosted | Free tier + paid | 250+ models | Yes | Yes | Yes |
Compiled from public docs and vendor sites; verify current pricing with the vendor. No vendor pays for placement. Each tool links to a fuller profile.
How to choose
Use OpenRouter for instant access to hundreds of models through one key and account. Use LiteLLM or Portkey when you want an open-source, self-hostable proxy with fallbacks, budgets, and logging. Cloudflare and Kong gateways fit teams standardizing on those platforms. Check how many providers are supported and whether caching and budget controls meet your needs.
FAQ
What problem does an AI gateway solve?
It centralizes access to multiple model providers behind one API, adding reliability (fallbacks, retries), cost control (caching, budgets), and observability (logging) without changing app code per provider.
Does a gateway add latency?
A small proxy hop, usually a few milliseconds, which is typically negligible next to model inference time — and caching can make repeated calls dramatically faster.
Can I self-host a gateway?
Yes. LiteLLM, Portkey's gateway, and Kong's AI gateway are open-source and self-hostable; OpenRouter and Cloudflare are managed services.