LLM Cost Calculator 2026: Cost per Task, Not Cost per Token

The useful unit for an LLM bill is not one million tokens. It is one completed task: one support ticket answered, one invoice extracted, one pull request reviewed, one internal question resolved. Per-token prices are only the price list. Cost per task is the invoice reality after retries, tool calls, context, cache hits, routing, batch discounts, and failed outputs. This is the calculator we use before optimizing an AI product, because the model swap is rarely the first lever.

Engineering perspective, not vendor pricing advice. The formulas below are stable; the example prices are directional and must be replaced with your current provider prices before a commit. The discount mechanics are current as of July 8, 2026: OpenAI's Batch API lists a 50% discount and a 24-hour completion window, OpenAI prompt caching starts at 1,024 prompt tokens, Anthropic cache reads are priced at 0.1x base input tokens and its Batches API charges 50% of standard prices, and Gemini's implicit caching is enabled by default for Gemini 2.5 and newer models. Re-check the linked primary sources before budgeting.

The calculator in one formula

Start with one task, not one API call. A task may contain multiple model calls, a retrieval step, tool calls, a verifier, and sometimes a retry. The all-in cost is:

In short:

Cost per completed task = model cost for all calls + retrieval and infra + retry cost + human rework cost.

That last line is why we wrote cost per token vs cost per task. A cheaper model that needs more turns, creates longer outputs, or fails more often can be more expensive than the model with the scary price list.

Which inputs belong in your calculator?

If your spreadsheet has only input tokens, output tokens, and model price, it is undercounting. Add these fields before touching the stack:

• Task volume. Count real business units: tickets, documents, quotes, pull requests, checks, research briefs. "Chat messages" is usually too vague.
• Calls per task. Agentic workflows often spend the bill in loops. One user request can become classification, retrieval, answer draft, tool call, verifier, rewrite, and audit log summary.
• Average input per call. Split stable prefix, retrieved context, conversation history, tool schemas, and volatile user data. This is where caching and context compression show up.
• Average output per call. Reasoning and coding agents can be output-heavy. A model with cheap input and expensive output may look cheap until it starts writing thousands of tokens.
• Cache hit rate. Measure cached tokens, not requests. OpenAI exposes `cached_tokens`; Gemini exposes cached token counts; Anthropic separates cache write and cache read pricing.
• Batchable share. Anything that can wait belongs in batch first: evals, offline extraction, enrichment, classification, summarization, migration jobs.
• Escalation rate. If a cheap model handles first pass and a stronger model handles hard cases, the percentage escalated is a product KPI.
• Quality floor. Put the pass rate from your eval harness next to cost. Without that, you are comparing bills and ignoring whether the work still works.

"If your calculator cannot show what one successful task costs, it is not an AI cost calculator. It is a token receipt."

Worked example: support triage

Imagine a support triage workflow that reads a ticket, retrieves policy snippets, drafts an answer, and asks a verifier whether the answer is grounded. The naive spreadsheet says: one answer, maybe 4,000 input tokens and 600 output tokens. The production trace says something else:

The model cost is not the answer call. It is 3.18 average calls, plus retrieval, plus a retry tail. If 60% of the draft input is a stable system prompt, policy framing, and tool schema, prompt caching can matter more than switching models. If only 20% of tickets need the strong model, routing matters more than shaving a few cents off the default model. This is the same pattern behind our LLM token cost reduction playbook, but expressed as a calculator.

Where prompt caching enters the math

Prompt caching is the first lever because it changes the cost of repeated input without changing the model or the answer. Use this version of the formula:

Input cost = uncached_input * normal_input_price + cached_input * cache_read_price + cache_writes * cache_write_price.

The operational trick is boring and worth money: put the stable content first and the volatile content last. OpenAI's prompt caching guide says caching is available for prompts of 1,024 tokens or more and recommends static or repeated content at the beginning. Anthropic's guide prices cache reads at 0.1x base input tokens, while cache writes are more expensive than normal input, so the write only pays back when the prefix is reused. Gemini's current Interactions API page says implicit caching is on by default for Gemini 2.5 and newer models and recommends putting large common content at the beginning.

• Good cache prefix: system instruction, tool definitions, output schema, product policy, stable retrieval context.
• Bad cache prefix: timestamp, user ID, random trace ID, per-request document, current ticket body.
• Metric to track: cached input tokens divided by total input tokens, by feature and model.

Primary docs: OpenAI prompt caching, Anthropic prompt caching, and Gemini context caching.

Where batch APIs enter the math

Batching is not a latency optimization. It is a price and throughput optimization for work that can wait. The calculator should split traffic into synchronous and asynchronous work:

Total model cost = live_cost + batchable_cost * batch_multiplier.

OpenAI's Batch API documents a 50% cost discount versus synchronous APIs and says batches complete within 24 hours, often faster. Anthropic's Message Batches API page says it reduces costs by 50%, with most batches finishing in less than one hour and results available after all messages complete or after 24 hours. That makes batch the default for evals, nightly document enrichment, offline extraction, migration scripts, moderation backfills, and analytics summarization.

Do not batch user-facing chat unless the product can tolerate delay. Do batch your eval harness. Many teams spend real money continuously re-testing prompts and model candidates, then forget that those tests do not need live latency. We covered why this matters in when an LLM eval pays for itself.

Primary docs: OpenAI Batch API and Anthropic batch processing.

Routing: the cost lever with a quality trap

Routing changes the model mix per task. The calculator needs a strong-model share, not just one model price:

Routed cost = cheap_path_cost * (1 - escalation_rate) + strong_path_cost * escalation_rate + verifier_cost.

The public RouteLLM project frames this well: route simpler queries to cheaper models, keep stronger models for harder queries, and calibrate the threshold on traffic similar to your own. Its README reports up to 85% cost reduction while maintaining 95% GPT-4 performance on benchmarks. Treat that as a research reference, not your production number. Your traffic can route very differently, especially if your product has domain-specific edge cases.

The safe routing pattern is:

1. Cheap default. Start with the cheapest model that passes the easy majority.
2. Verifier. Check schema validity, grounding, policy compliance, and confidence.
3. Escalation. Send uncertain, high-stakes, or failed cases to a stronger model.
4. Eval gate. Compare cheap path, strong path, and routed path on real examples before changing production traffic.
5. Escalation monitoring. If the strong-model share rises, either traffic changed or the cheap model is being overworked.

This is where LLM gateways and routers help. LiteLLM, Portkey, OpenRouter, or a custom RouteLLM layer can centralize logs, model mix, fallback, budgets, and routing. The calculator is the reason to install that layer; the gateway is how you measure it.

Research and tooling: RouteLLM, RouteLLM paper, batch-level query routing under cost and capacity constraints, and routing with batch prompting.

Local models and self-hosting: put utilization in the formula

Self-hosting does not replace token math with free inference. It replaces variable token cost with GPU cost, utilization risk, ops time, redundancy, and eval upkeep. Use this formula before buying the machine:

Self-host cost per task = (gpu_hour_cost + ops_hour_cost + redundancy + monitoring + eval_upkeep) / completed_tasks_per_hour.

The denominator decides everything. A GPU that is 80% loaded all day can make local inference look rational. A GPU that idles at 12% because traffic is bursty turns into a very expensive badge of sovereignty. This is why our EU self-hosting cost guide starts with volume and data residency, not with GPU specs.

For most early and mid-stage products, self-hosting wins in two cases:

• Data residency or governance forces it. If the data cannot leave your infrastructure, cost becomes the second question.
• High, steady volume fills the hardware. Against cheap hosted open-weight APIs, the break-even usually requires sustained volume, not occasional peaks.

The model choice belongs after that. Our open-weight LLM comparison covers DeepSeek, Qwen, Kimi, GLM, and Llama from a European deployment perspective. Your calculator should include the model's pass rate on your eval, not only its tokens per second.

Context compression and semantic caching

After prompt caching, batch, and routing, the next question is whether the model needed the context at all. Split this into two lines:

• Semantic caching. If a new request is close enough to a previous request, return the previous answer or a lightly adapted answer. This can cut cost hard in repetitive support and internal assistant workloads, but it can leak stale or unauthorized answers if permissions and invalidation are sloppy.
• Context compression. Send the smallest correct context to the model. For coding and agent workflows, that means summaries, file maps, relevant snippets, and tool output pruning instead of re-sending the entire workspace on every turn.

This connects directly to why coding agents fail on context, not intelligence and to our note on text-as-image token savings. Compression is powerful, but exact values, IDs, amounts, hashes, legal clauses, and permissions must stay exact. If the optimization is lossy, the calculator needs a failure-cost line.

A spreadsheet layout you can copy

Set up one row per task type, not one row per model:

Once the table exists, optimization becomes obvious. High cached-token opportunity? Reorder prompts. High batchable share? Move jobs off live endpoints. High escalation rate? Improve the cheap path or accept that the task is hard. High retry rate? Fix reliability before arguing about model price. High human rework? The cheaper model is probably lying to your spreadsheet.

What order should you optimize in?

Do the cheap, low-risk changes before the architectural ones:

1. Instrument cost per task. Log task ID, model, tokens, cache hits, retries, latency, result status, and eval verdict.
2. Take caching seriously. Stable prefix first, volatile data last, track cached tokens by feature.
3. Batch offline work. Evals and enrichment should not pay live prices.
4. Route with a verifier. Cheap default, strong fallback, monitored escalation rate.
5. Right-size the model. Test open-weight and smaller models on your own eval set.
6. Compress context. Remove irrelevant history and repeated workspace context.
7. Self-host only when volume or governance forces it. Price utilization and engineers, not just GPU hours.

The center of gravity is the eval harness. Every cost lever can quietly reduce quality. A calculator without evals will celebrate the cheapest path even when it creates bad answers. A calculator with evals shows the real trade: cost per successful task.