When Local Models Beat APIs: A Break-Even Calculator for EU Companies

Local LLMs beat APIs only when the workload, governance, and operating model line up. A cheap GPU hour is not a business case. The business case is cost per successful task after utilization, concurrency, redundancy, engineer time, eval upkeep, and EU data residency are counted. If the open-weight model does not pass your eval, the calculator stops. If it passes but the GPU is idle most of the day, the calculator probably stops too.

This article is the self-hosting companion to our LLM cost calculator 2026. That post compares API bills by task. This one compares API tasks against local inference. The hard lesson from 2026 utilization research is simple: the same H100 can look extremely cheap or painfully expensive per million output tokens depending on offered load and concurrency. A recent arXiv paper measured a spread from $0.21 to $15.25 per million output tokens on identical H100 hardware when utilization changed, and argues that utilization-naive calculators understate cost by 1 / U. Treat that as the center of the model, not a footnote.

The short answer

Use an API when traffic is low, bursty, fast-changing, or dependent on frontier reasoning. Use a local model when you have high and steady volume, a model that passes your eval, a team that can operate the stack, and either strict EU data-residency requirements or an API alternative expensive enough to leave real margin. Hosted EU-region APIs and managed open-weight endpoints sit in the middle: often the best answer before full self-hosting.

The calculator

Start with the API baseline. The right comparison is not provider invoice versus GPU invoice. It is API cost per successful task versus local cost per successful task.

The local inference line needs one extra formula:

Effective local cost per 1M tokens = ((gpu_hourly_rate + infra_hourly + ops_hourly) / (tokens_per_second * 3600 * measured_utilization)) * 1,000,000.

Measured utilization is the trapdoor. If your spreadsheet assumes 80% utilization but production gives you 12%, your cost is not a little wrong. It is wrong by a factor of roughly 6.7 before redundancy, on-call, and eval upkeep enter the room.

Inputs to collect before deciding

Do not start with model weights or GPU quotes. Start with traces from the task you want to replace.

Why utilization beats GPU price

GPU quotes are seductive because they are simple. Utilization is annoying because it is an emergent property of your traffic. A single interactive assistant may have long idle periods and sudden bursts. A nightly extraction job can run at a controlled queue depth for hours. The second workload can make a local model look rational; the first often cannot.

The concurrency-aware view uses Little's Law: in-flight requests are driven by request rate and latency. That matters because LLM serving is not a static token factory. Prefill, decode, KV-cache memory, batching, queueing, and output length all shape throughput. The 2026 cost-estimation paper is useful because it points at the hidden assumption in most calculators: users type a utilization value into the sheet, but the workload determines whether that value is plausible.

Serving stack choices matter too. The 2025 vLLM versus HuggingFace TGI study found vLLM up to 24x higher throughput in high-concurrency workloads, while TGI showed lower tail latency in interactive moderate-concurrency scenarios. That does not mean vLLM always wins. It means your calculator needs a benchmark row for the exact workload, model, quantization, context length, and latency SLO.

Do not compare against the wrong API

A local model does not need to beat the most expensive frontier model in the provider catalogue unless that is the model your task truly needs. It needs to beat the cheapest acceptable managed path after caching, batching, routing, and model right-sizing.

• OpenAI pricing separates input, cached input, and output prices, so stable prefixes can materially change the baseline before self-hosting enters the picture.
• Gemini pricing includes context caching and batch pricing on paid tiers, and the page explicitly separates whether prompts are used to improve products by tier.
• Amazon Bedrock says select foundation models are available for batch inference at 50% lower price than on-demand inference.
• Azure OpenAI adds another comparison point for EU companies because data zones, provisioned throughput, reservations, and enterprise procurement can matter as much as token price.

Only after those levers are priced should self-hosting enter the model. For the optimization ladder, see how to reduce LLM token costs in 2026. For model selection, see our open-weight LLM comparison.

A worked break-even example

Imagine an EU SaaS company running a document-enrichment pipeline. The current API path costs 0.018 dollars per successful document after caching, batch discounts, retries, and verifier calls. The local candidate passes evals at the same quality bar. The self-hosting stack costs 9,800 dollars per month all-in: GPU lease, hot spare capacity, storage, logging, monitoring, networking, engineering time, and eval maintenance. Variable local cost is 0.003 dollars per document.

Add a risk buffer. If the eval pass rate drops, if utilization is lower than the benchmark, if failover requires a second hot GPU, or if demand is seasonal, the break-even moves right. If the workload is async and can saturate the GPU overnight, it moves left. If the API baseline is a cheap open-weight hosted endpoint rather than a frontier model, it may move so far right that self-hosting is not worth the distraction.

EU-specific factors

For EU companies, the local-versus-API question is rarely only about price. GDPR, data processing agreements, customer procurement, sector rules, and auditability can decide the shape of the architecture. But governance still has multiple options.

We go deeper on the governance side in EU data residency for AI apps and the broader economics in what self-hosting LLMs really costs in the EU.

The local stack you are actually pricing

The stack is not "model plus GPU". A production local deployment usually includes a serving runtime such as vLLM, model artifact management, quantization decisions, autoscaling or queue control, telemetry, prompt and output logging policy, access control, eval jobs, rollout gates, incident response, and a fallback route when the local model fails or the GPU pool is saturated.

That fallback route matters. The most pragmatic architecture is often hybrid: local handles cheap, stable, high-volume work; an API handles hard cases, overflow, image/audio features, tool-heavy reasoning, or the tasks where frontier quality still wins. At that point the calculator becomes a routing calculator: local default, API fallback, measured escalation rate, and evals that prove the cheap path is still acceptable.

Decision checklist

1. Can the local model pass the eval? If no, stop.
2. Have you priced the optimized API baseline? Include prompt caching, batch, routing, smaller models, and provider-specific EU options.
3. Can the workload keep hardware busy? Use measured concurrency and latency, not hoped-for utilization.
4. Can your team operate it? Add serving, security, monitoring, upgrades, rollback, and on-call time.
5. Does governance require it? If data cannot leave your controlled environment, cost is secondary but still measured.
6. Is there a hybrid path? Local for steady cheap work, API for overflow and hard tasks is often the actual optimum.

Sources and live-price caveat

Provider prices and model names move quickly. The formulas above are stable; example price mechanics are a July 2026 snapshot. Re-check OpenAI pricing, Gemini API pricing, Amazon Bedrock pricing, and Azure OpenAI pricing before budgeting. For utilization and serving-system assumptions, read Beyond Per-Token Pricing and the vLLM versus TGI performance study.