LLM Gateways Compared 2026: LiteLLM vs OpenRouter vs Portkey vs RouteLLM

Most teams wire their product straight to one provider's SDK. It works until it doesn't. Then the provider has an outage and your app goes down with it. Then finance asks why one runaway job burned a month of budget in an afternoon. Then a new model ships that is cheaper and better for half your traffic, and switching means touching every call site. So the team starts bolting on retries, a spend cap, a second provider, a cache, and within a quarter you have built half an agent infrastructure layer badly, in your own codebase, with no one owning it.

That missing layer has a name: the LLM gateway. One endpoint in front of many providers can centralize fallback, caching, spend limits, and routing. This is an engineering perspective, not a vendor pitch. We re-checked the feature, licensing, privacy, and pricing claims below against first-party sources on 2 September 2026. Product catalogs and fees change, so verify them again during procurement. The reference points come from Wavect's AI product work, where we have put a gateway in front of production traffic and lived with the tradeoffs.

What does an LLM gateway actually do?

A gateway is a proxy that sits between your application and the model providers. Your code calls one endpoint, usually in the OpenAI request format, and the gateway translates and forwards the call to whichever provider should serve it. That single seam is where you get the features you would otherwise rebuild by hand:

• One endpoint over many providers. A common request surface reduces provider-specific code. It does not erase lock-in because tools, schemas, safety behavior, and model semantics still differ.
• Fallback. When a provider returns an eligible error or times out, a configured gateway can retry another deployment, model, or provider. That improves resilience but does not guarantee availability.
• Caching. Identical or semantically similar requests can return a stored response and skip the model call entirely, which cuts both cost and latency on repetitive traffic.
• Spend limits and keys. Per-key, per-user, or per-team budgets and rate limits, so one bad loop cannot drain the account, and so you can hand a scoped virtual key to each team.
• Routing. Send the easy majority to a cheap model and the hard minority to a frontier one, on rules or on a learned policy.
• Observability. One place to see every request, its cost, its latency, and its tokens, broken down by model, key, and feature.

Routing is the cost lever most people come for, and we covered the economics of it in how to cut LLM token costs in 2026. This post is about the tools that give you that lever plus the rest of the layer.

How do LiteLLM, OpenRouter, Portkey, and RouteLLM differ?

These four names come up together but they are not the same kind of thing. LiteLLM and Portkey offer gateway software, OpenRouter is a hosted aggregator, and RouteLLM is a routing research framework. Here is the shape of each, verified against first-party sources on 2 September 2026.

LiteLLM documents an OpenAI-compatible gateway and SDK for 100+ LLM APIs, with virtual keys, spend tracking, routing, caching, and logging. Its repository is mostly MIT-licensed, while files under its enterprise directory use a separate commercial license, so check the exact feature boundary before standardizing. OpenRouter is a hosted aggregator with consolidated billing, provider routing, and a large catalog. Portkey's gateway is MIT-licensed and supports routing, caching, retries, guardrails, and self-hosting; Portkey also sells hosted and enterprise plans. RouteLLM is different in kind: an LMSYS and Berkeley framework for serving and evaluating learned routing decisions, not the surrounding gateway control plane.

"Three of these are gateways and one is a router. Comparing RouteLLM to LiteLLM is comparing the brain to the body. Most teams need both, and most reach for the body first."

Self-host or hosted: which should you pick?

This is the first real decision, and it usually decides the tool. The tradeoff is control and data residency against operational burden.

If LiteLLM is already your likely choice, use our production guide to self-hosting LiteLLM for the Postgres, Redis, key-management, patching and go-live controls that sit behind the word self-hosted.

• Hosted (OpenRouter, Portkey cloud). You avoid operating the gateway, but add another processor to the request path. OpenRouter says prompt and completion logging is opt-in, while request metadata is retained and upstream-provider policies still apply. Its privacy controls include provider-policy filters, and enterprise customers can request EU in-region routing. Its FAQ lists pass-through inference rates, a 5.5% credit-purchase fee with a $0.80 minimum, and separate BYOK terms. OpenRouter's pricing page now expresses some BYOK allowances by plan, so reconcile the FAQ, pricing page, and contract before budgeting.
• Self-host (LiteLLM, Portkey gateway, RouteLLM). You control the gateway deployment and upgrade cadence, but calls still reach whichever upstream model provider you configure. LiteLLM's open-source components do not add a per-request gateway fee, while commercial features have separate terms. Persistent spend tracking and production topologies may add Postgres, Redis, and other services; those are design choices, not requirements for every installation.

A practical default is to start with the deployment model that already satisfies your data-processing, reliability, and operations constraints. Self-hosting is not automatically compliant, and hosted is not automatically unsuitable: map subprocessors, retention, regions, keys, and incident ownership before deciding. That mirrors the build-versus-buy logic we apply across AI product engagements, including work like Twinsoft AI.

How good is the cost tracking and observability?

Provider dashboards can show account-level usage, but a gateway can consolidate attribution across providers and attach application metadata. The four tools cover that job at different depths.

• LiteLLM documents multi-tenant spend tracking, budgets, rate limits, and logging callbacks. Self-hosting controls where gateway telemetry is stored, but not the retention behavior of configured upstream providers.
• Portkey offers logs, traces, analytics, feedback, and guardrails. Its September 2026 pricing meters recorded logs by plan, with 10,000 monthly logs on Free and 100,000 on the $49 monthly Production plan before listed overages. Requests continue if the Free logging allowance is exceeded, but additional logs are not recorded.
• OpenRouter gives you a usage dashboard and analytics across the models you call, which is enough for many teams and requires zero setup.
• RouteLLM includes serving and evaluation utilities, but not the multi-tenant spend, key, and production-telemetry control plane offered by a gateway.

One caution we always add: spend dashboards tell you what you paid, not whether quality held. Some platforms now include evaluation features or integrations, including Portkey's eval templates and batch-evaluation guidance, but you still need a task-specific acceptance set and release gate. A generic dashboard cannot prove that your routed workload remains good enough.

Where does RouteLLM fit, and are the numbers real?

RouteLLM is the only one of the four that is purely about the routing decision: given a query, send it to the strong model or the cheap one. The published figures are genuinely strong, and worth citing carefully. The LMSYS and Berkeley team report that, with augmented training data, their matrix-factorization router reaches about 95% of GPT-4's performance while sending only 14% of calls to the strong model, which they put at roughly 75% cheaper than a random baseline, and over 85% cost reduction on the MT Bench evaluation.

Read those numbers as directional and as the source frames them: they come from the original RouteLLM paper, benchmarked on specific datasets (MT Bench, MMLU, GSM8K) against a GPT-4-class strong model, published 2024 and presented at ICLR 2025. Your traffic is not those benchmarks, so the savings on your workload will differ. The honest takeaway is the shape, not the exact percentage: a learned router can hold most of the quality while sending a minority of calls to the expensive model. You still have to prove it on your own eval before you trust it.

In practice you do not choose RouteLLM instead of a gateway. You can run RouteLLM as the routing brain and put a gateway around it for fallback, keys, caching, and observability, or you use a gateway's own simpler routing rules and skip the framework. RouteLLM earns its place when routing is your single biggest cost lever and rule-based routing is leaving savings on the table.

Evaluating OmniRoute specifically? Keep that product question separate from the category comparison. Our OmniRoute AI routing setup and production checklist covers route design, access hardening, failure drills and the eval gates to run before rollout.

How should you choose?

Map the tool to the constraint that is actually binding you, not to the longest feature list:

1. Need it today, no infra to run. Reach for a hosted option. OpenRouter for breadth of models behind one key, Portkey cloud if you also want observability and guardrails from day one.
2. Data residency or full control matters. Self-host. LiteLLM if you want a lean, widely-used open-source proxy; Portkey's open-source core if observability and guardrails are first-class requirements.
3. Observability and guardrails are the priority. Portkey is built around them. The others log; Portkey makes the log the product.
4. Routing is your biggest cost lever. Add RouteLLM as the routing brain inside whichever gateway you picked, and prove the savings on your own eval.
5. You are unsure. Start with one hosted gateway, instrument it, and let two weeks of real spend-by-feature data tell you what to optimize. The data answers the question faster than the comparison table.

Keeping a narrow OpenAI-compatible seam can reduce switching work, but it does not make gateways interchangeable. Test streaming, tool calls, structured outputs, errors, authentication, and provider-specific extensions before a migration.