Smarter Token Usage: Measure Cost per Accepted Coding Task

Back

Kevin Riedl

11 min read · 17 Aug 2026
Last reviewed October 5, 2026

Next

A coding agent can send fewer tokens and still produce a larger bill. Cached reads, fresh input, cache writes and output have different prices. A cheaper-looking run can also transfer work to a second model or a human reviewer. The useful question is: how much did one accepted change actually cost?

Reviewed October 5, 2026. This article is a measurement and rollout guide for an existing coding workflow, not a ranking of the cheapest models. The worked example below is synthetic accounting data, not a Wavect benchmark or a promised saving.

1. Normalize the bill before changing the agent

Log task ID, attempt ID, model/version, provider, token categories, prices, tool charges, latency and acceptance result. Include rejected attempts and auxiliary agents. A failed attempt still belongs in the numerator; accepting the same task twice must not increase the denominator.

Anthropic reports uncached input_tokens separately from cache creation and cache reads. Split writes by their five-minute or one-hour lifetime. Standard write multipliers are 1.25× and 2×. Cache reads are usually 0.1× base input, but Opus 5.5 uses 0.05× and Fable 5.1/Mythos 5.1 use 0.025×. These are model-specific billing categories, not one universal token price.

Do not reuse the same parser for every provider. In OpenAI usage, total input includes cached input; models exposing cache_write_tokens require writes to be separated too. Derive ordinary input from the documented fields without counting it twice. Missing or inconsistent usage is an accounting error to investigate, not automatically zero cost. Apply output and reasoning charges according to the provider’s billing rules.

2. Make repeated context cacheable, then measure it

Keep reusable instructions and tool contracts stable, with changing task material later in the prefix where the API permits. Confirm model-specific cache minimums, expiry and actual read/write counters. Claude Code can rebuild a cache after changes or inactivity, and subagents may have a different prefix. A longer lifetime has a higher write cost and is not automatically cheaper.

3. Route and compress behind real acceptance checks

Evaluate routing on a fixed set of repository tasks with deterministic tests and review criteria. Let a cheaper model handle the task classes it demonstrably passes. Send high-risk changes directly to stronger verification or human review when appropriate. A model’s self-reported confidence or valid JSON is not proof that a code change is correct. RouteLLM is one implementation reference, not evidence of guaranteed savings on your repository.

Trim irrelevant logs, retrieve only the necessary files, and keep a compact task manifest. Preserve requirements, file versions, failure evidence and a way to retrieve the full material. Measure summarizer calls, lost cache reuse and subsequent retries. Shorter context does not guarantee lower cost or higher accuracy; the third example below deliberately shows a cache regression.

For one concrete implementation, our RTK AI 1.0 review separates shell filtering, optional pipelines and API proxying, including the license split and the limits of its savings statistics.

4. Replace deterministic work and batch independent jobs

Use formatters, parsers, search and narrowly scoped scripts for deterministic work. Measure which calls disappear rather than claiming a fixed percentage. Keep permission checks and review for destructive operations. Batch separate evaluations or classifications when the latency budget permits; do not batch a live agent step that depends on the previous response. OpenAI documents a 50% batch discount and a 24-hour completion window for supported requests, not a universal discount for every provider or workflow.

5. Reproduce the difference between tokens and cost

Use the same two bug-fix tasks and the same acceptance checks for every scenario. Each illustrative scenario contains three attempts, including one rejection, and two unique accepted changes. Token counts aggregate all attempts. The rates are Sonnet 5.5 standard API prices reviewed on October 5: USD 2 uncached input, 2.50 five-minute writes, 4 one-hour writes, 0.20 reads and 10 output per million tokens. No batch or regional modifier is assumed.

For this example only, add 30 total review minutes at an assumed USD 60/hour plus USD 0.40 of other workflow costs to each scenario. Those are arithmetic assumptions, not measured labor rates or tool bills. In production, replace them with actual review, compression-worker, infrastructure and tool costs; do not count worker tokens twice.

The cache-regression scenario sends fewer tokens than the optimized scenario but costs more at the API. Including review also changes the economic scale of the comparison. With zero accepted tasks, cost per accepted task is undefined: report the failed run and its spend, rather than dividing by zero or reporting a saving.

Run this accounting fixture with Python 3.10 or later. It checks the arithmetic, not agent quality. Replace the counters and rates before using it for your own bill.

from decimal import Decimal as D

# USD per million tokens: uncached, write 5m, write 1h, read, output.
RATES = tuple(map(D, ("2", "2.50", "4", "0.20", "10")))
CASES = {
    "baseline": (1_000_000, 200_000, 0, 8_000_000, 100_000),
    "optimized": (500_000, 200_000, 0, 4_000_000, 100_000),
    "cache_regression": (1_400_000, 0, 0, 0, 100_000),
}

def cost(tokens, accepted=2, review_minutes=30, other_usd="0.40"):
    if type(accepted) is not int or accepted <= 0:
        raise ValueError("accepted must be a positive count of unique tasks")
    if len(tokens) != len(RATES) or any(type(n) is not int or n < 0 for n in tokens):
        raise ValueError("provide five non-negative integer token counts")
    review, other = D(str(review_minutes)), D(str(other_usd))
    if not review.is_finite() or not other.is_finite() or min(review, other) < 0:
        raise ValueError("review time and other costs must be finite and non-negative")
    api = sum(D(n) * rate for n, rate in zip(tokens, RATES)) / D(1_000_000)
    review_usd = review / D(60) * D("60")
    return api, (api + review_usd + other) / D(accepted)

for name, tokens in CASES.items():
    api, per_accepted = cost(tokens)
    print(f"{name}: API=${api:.2f}; total/accepted=${per_accepted:.2f}")

6. Roll out one change at a time

1. Capture a baseline before changing prompts or model routes.
2. Change one variable: prefix stability, retrieval, routing, a deterministic tool or an eligible batch lane.
3. Replay the same tasks and record failures, accepted changes, review time and the full bill.
4. Keep the change only when quality and latency remain acceptable and the measured cost improves.

For the broader cost strategy, use How to Cut LLM Token Costs; for architecture, use the multi-model coding-agent guide. The Spotify shunt review separates main-agent context reduction from worker costs and verification effort.

Primary sources reviewed October 5, 2026

• Anthropic: cache billing and model-specific rates
• Claude Code: cache lifetime, prefix changes and usage counters
• OpenAI: prompt-cache accounting and explicit writes
• RouteLLM: routing implementation and evaluation
• OpenAI: Batch API eligibility, price and completion window