--- title: "AI Coding Agents 2026: Context Was the Bottleneck" canonical: https://wavect.io/blog/ai-coding-agents-context-not-intelligence/ language: en description: "Rakuten ran a coding agent for 7 hours on vLLM at 99.9% accuracy. The real lesson: context, not intelligence, was the bottleneck. How to direct, review, and gate agents." image: "https://wavect.io/img/blog/headers/header_ai-coding-agents-context-not-intelligence.png" --- [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 9 min read · 23 Jun 2026 [**Next**](/blog/internal-ai-adoption-2026/) # The Bottleneck in Software Was Never Intelligence. It Was Context. TL;DR Rakuten pointed a coding agent at vLLM and had it implement a specific method, running about seven hours with occasional human guidance and hitting 99.9% numerical accuracy against the reference. The viral "12.5 million lines of code" figure is roughly twenty times too high; the real point is that the agent navigated a large, unfamiliar, multi-language codebase and landed the change in the right place. The lesson is that the bottleneck was never intelligence, it was context: humans tire and make mistakes holding a big system in their heads, and agents do not get that context fatigue. This does not erase engineers, it moves the work. The ones scaling are good at directing agents, feeding them the right context, gating output with tests and guardrails, and reviewing like a senior to catch the confident wrong answer. Delegation became the new deep work. One story keeps getting passed around: Rakuten handed [an AI coding agent](/glossary/llm/) a single task inside a large open-source project, walked away, and came back hours later to working code. It is a good story. It is mostly true. And the part everyone repeats, the hours and the accuracy number, is the least interesting part of it. The interesting part is why it worked at all, and what it says about the job of engineering now. This is an engineering perspective, not a vendor pitch. We direct coding agents on real client work every week, so this is less a hot take than a description of how the work actually changed. ## What actually happened at Rakuten? Here are the facts, stated carefully, because the viral version inflates one of them. At its [Claude Opus 4 launch in May 2025](https://www.anthropic.com/news/claude-4), Anthropic said Rakuten had validated the model with a demanding open-source refactor that ran independently for about seven hours. The detail came later, in [Rakuten's own customer story](https://claude.com/customers/rakuten): an engineer pointed Claude Code at [vLLM](https://github.com/vllm-project/vllm), the open-source inference engine, and asked it to implement a specific activation vector extraction method. The result reportedly hit 99.9% numerical accuracy against the reference implementation. Two honest corrections, because we would rather be right than dramatic: - It was not unattended. The engineer described providing occasional guidance across those hours. "Ran independently for seven hours" is real and impressive. "Fully autonomous, zero humans" is not what happened. - 99.9% is a narrow metric. It is the numerical accuracy of one method's output against a reference, not a general statement that the agent is 99.9% correct at everything. Useful, specific, and easy to misread as a headline. And the number the internet loves most, that vLLM is "12.5 million lines of code," is the one to be most skeptical of. It appears in Rakuten's own copy, so people quote it in good faith, but it is off by roughly twenty times. The vLLM core is on the order of 84,000 lines of Python, and under 600,000 lines across all languages. You only reach 12.5 million by counting things nobody reasons about: full git history, every related repository, vendored and generated kernels. The real point survives the correction cleanly: Claude navigated a large, multi-language codebase it had never seen, in Python, CUDA, and C++, and landed a change in the right place. That is the impressive thing. It was never the line count. ## Why was context the real bottleneck, not intelligence? Strip the demo down and the lesson is almost boring. A senior engineer who already knew vLLM could have written that method too. A new hire could have, eventually, after weeks of wrong turns. The difference between the senior and the new hire was never raw intelligence. It was how much of the codebase each one could hold in their head and reason about at once. That is the bottleneck in most software work. Not "can a smart person figure this out," but "can anyone hold enough of this system in working memory to make the right change in the right place without breaking three others." On a large codebase that is genuinely hard, and it is hard in a way that has nothing to do with how clever you are. ## What does "AI doesn't get context fatigue" actually mean? Humans are bad at holding large context for long stretches, and not because we are dim. We tire. We forget the thing we read forty files ago. We take a break and lose the thread. We make small mistakes late in a long session that we would never make in the first hour. Holding a sprawling mental model of a system is exhausting, and exhaustion is where the bugs come from. A coding agent does not get tired in that way. It can keep the relevant slice of a large codebase in front of it and reason about it consistently at hour seven the same way it did at minute ten. That is the actual unlock in the Rakuten story. Not that the model is smarter than a senior engineer, but that it does not degrade across a long, context-heavy task the way a person does. Sustained context, not superior intelligence. The catch, and it is a real one, is that the agent only reasons well over the context it is actually given. Point it at the wrong files, or starve it of the constraints that matter, and it will confidently build the wrong thing without fatigue. Feeding it the right context is now the skill. We wrote about the cost side of that discipline in [how to cut LLM token costs in 2026](/blog/reduce-llm-token-costs-2026/): managing context, not just spending tokens, is most of the game. For a concrete architecture that separates durable memory, shared state, bounded reasoning and intent routing, see our [fact-checked Meterless context-layer review](/blog/meterless-ai-agent-context-layer-review/). When cross-file relationships are the missing layer, our [Graphify buyer review](/blog/graphify-review-codebase-knowledge-graph/) compares a codebase knowledge graph with repository search and RAG, then gives CTOs a measured two-week adoption test. When parallel agents make repository setup and change integration the next constraint, use our [Git worktrees vs Jujutsu decision guide](/blog/git-worktrees-vs-jujutsu-ai-coding-agents/) to choose a cached Git baseline or a measured Jujutsu pilot. For the tooling layer that writes that context down between sessions, see our [Graft review](/blog/graft-review-agent-repo-map/) on whether an agent repo map belongs in git or stays a rebuildable local cache. ## Does this make engineers disappear? No, and the people predicting that are usually selling something. What it does is move the work. The engineers who are scaling right now are mostly not the ones writing every line by hand. They are the ones who got good at directing agents, reviewing output critically, and knowing the moment an agent is about to do something stupid. That last skill is underrated. An agent will produce confident, well-formatted, plausible code that is subtly wrong, and a junior reviewer will wave it through because it looks right. Catching that requires exactly the judgment that years of writing code by hand builds. The experience does not become worthless. It changes from "I type the solution" to "I recognize the wrong solution before it ships." If you want a concrete version of what that review looks like, our [vibe-code production-readiness checklist](/blog/vibe-code-production-readiness-checklist/) is the list we actually run agent output against. ## What does good engineering look like when agents write the code? Delegation became the new deep work. The deep, valuable hours used to be the ones spent heads-down writing the hard function. Increasingly they are the ones spent scoping a task precisely, assembling the right context, and reviewing what comes back with a sharp eye. That is a genuinely different muscle, and a lot of strong engineers have not built it yet because for their whole career the bottleneck was typing the solution, not specifying it. Good engineering in this mode looks like: a tight, well-defined task; the right context handed to the agent up front; tests and guardrails that catch the wrong answer automatically; and a human who reviews like a senior, not a rubber stamp. The agent is fast and tireless. The engineer is the one who decides what "correct" means and verifies it actually got there. ![Kevin Riedl](/img/team/kevin.webp) "The agent does not get tired holding the whole codebase in its head. You do. That is the entire shift. Your job moved from writing every line to deciding what correct means and catching it when the agent gets it wrong." ## How should you restructure your workflow around coding agents? If you want the Rakuten outcome on your own work, the moves that matter, in order: 1. **Scope the task tightly.** "Implement this specific method, matching this reference" beats "improve the inference layer." A precise task is what made seven unattended hours possible. A vague one produces seven hours of confident wrong turns. 2. **Give it the right context, not all of it.** Point the agent at the files, interfaces, and constraints that actually matter. More context is not better; the correct context is. This is where most of the skill now lives. 3. **Gate with tests and guardrails.** The reason Rakuten could trust the output was a reference to check against. Reproduce that: a test suite, a reference, a guardrail that fails loudly when the answer is wrong. 4. **Review like a senior, not a rubber stamp.** Read the diff for the subtle, plausible-looking mistakes, the ones that compile and pass a shallow glance. This is the highest-leverage hour you will spend. 5. **Keep ownership and knowledge in-house.** An agent that ships code nobody on your team understands is a dependency, not a win. Make sure a human owns and can explain what shipped. None of this is exotic. It is the same discipline good engineering always needed, reweighted: less time producing the code, much more time specifying and verifying it. Semaprax is one research response to this context problem, not proof that a new language solves it. The [Semaprax benchmark](/semaprax/benchmarks/) freezes a small structured-context contract and publishes its limitations, including that it does not yet measure model tokens, task quality or repository-scale cost. ## Final thoughts The Rakuten story is not evidence that AI is smarter than your engineers. It is evidence that the bottleneck was never intelligence. It was context, and the human cost of holding a large system in your head without making tired mistakes. Agents do not get context fatigue, and that is the real shift. So the job moves. The engineers who scale are the ones who got good at directing agents, feeding them the right context, and catching the confident wrong answer before it ships. Delegation became the new deep work. The number to ignore is 12.5 million lines. The skill to build is knowing exactly what you are asking for, and recognizing when the agent is about to do something stupid. ## You may also like.. [**Programming Languages Matter Less. Software Engineering Matters More.** The evidence-backed follow-up: syntax and boilerplate get cheaper while system design, runtime literacy, memory safety, verification and maintainability gain value.](/blog/programming-languages-matter-less-ai/) [**AI Enablement vs a generic AI consultancy** One hands you a strategy deck. The other ships a working setup on your infrastructure that your team owns and can run.](/compare/ai-enablement-vs-generic-ai-consultancy/) Agent engineering ## Continue through this cluster Coding agents, MCP, context systems, evaluation and the controls required for dependable automation. [Start with the cornerstone**Graph Engineering for AI Agents: When Does a Knowledge Graph Pay Off?**](/blog/graph-engineering-ai-agents/) - [OpenViking Review 2026: Is Filesystem Memory Production-Ready?](/blog/openviking-agent-memory-review/) - [LLM-as-a-Verifier Explained: Architecture, Costs, and Production Fit](/blog/llm-as-a-verifier/) - [TrueForge Review: Is the Open-Source Agent Harness Production-Ready?](/blog/trueforge-agent-harness-review/) - [Agent-Readable Websites: llms.txt, Markdown Mirrors and What Breaks](/blog/agent-readable-website-llms-txt-markdown-mirrors/) - [Localized URLs Break hreflang: Keep One English Slug](/blog/english-slugs-vs-localized-urls-hreflang/) Inbox, without the noise ## Follow the work that matters to you Get a short email when we publish something new. Follow the whole blog or only the problems you care about. [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 9 min read · 23 Jun 2026 [**Next**](/blog/internal-ai-adoption-2026/) New posts by email × × Get new posts by email A short email when we publish. Free, no tracking. ## Structured Data ```json { "@context": "https://schema.org", "@graph": [ { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ], "employee": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "founder": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "legalRepresentative": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "name": "Wavect GmbH", "subjectOf": { "@id": "https://wavect.io/verified-claims.json#dataset", "@type": "Dataset", "creator": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "description": "A machine-readable registry of quantitative and qualitative claims published by Wavect, with review dates, localized page appearances and public third-party citations where available.", "inLanguage": "en", "isAccessibleForFree": true, "license": "https://creativecommons.org/licenses/by/4.0/", "name": "Wavect verified publication claims", "url": "https://wavect.io/verified-claims.json" }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "sameAs": [ "https://www.wikidata.org/wiki/Q139796367", "https://www.linkedin.com/in/jocr77/", "https://github.com/jo-chris" ], "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/#website", "@type": "WebSite", "inLanguage": [ "en", "de", "es", "zh" ], "name": "Wavect", "potentialAction": { "@type": "SearchAction", "query-input": "required name=search_term_string", "target": { "@type": "EntryPoint", "urlTemplate": "https://wavect.io/search/?q={search_term_string}" } }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/blog/ai-coding-agents-context-not-intelligence/#webpage", "@type": "WebPage", "dateModified": "2026-07-07", "inLanguage": "en", "isPartOf": { "@id": "https://wavect.io/#website", "@type": "WebSite" }, "lastReviewed": "2026-07-07", "url": "https://wavect.io/blog/ai-coding-agents-context-not-intelligence/" } ] } ``` ```json { "@context": "https://schema.org", "@type": "BlogPosting", "abstract": "Rakuten pointed a coding agent at vLLM and had it implement a specific method, running about seven hours with occasional human guidance and hitting 99.9% numerical accuracy against the reference. The viral \"12.5 million lines of code\" figure is roughly twenty times too high; the real point is that the agent navigated a large, unfamiliar, multi-language codebase and landed the change in the right place. The lesson is that the bottleneck was never intelligence, it was context: humans tire and make mistakes holding a big system in their heads, and agents do not get that context fatigue. This does not erase engineers, it moves the work. The ones scaling are good at directing agents, feeding them the right context, gating output with tests and guardrails, and reviewing like a senior to catch the confident wrong answer. Delegation became the new deep work.", "articleBody": " Blog overview/AI and agents/Agent engineering The Bottleneck in Software Was Never Intelligence. It Was Context. TL;DR Rakuten pointed a coding agent at vLLM and had it implement a specific method, running about seven hours with occasional human guidance and hitting 99.9% numerical accuracy against the reference. The viral \"12.5 million lines of code\" figure is roughly twenty times too high; the real point is that the agent navigated a large, unfamiliar, multi-language codebase and landed the change in the right place. The lesson is that the bottleneck was never intelligence, it was context: humans tire and make mistakes holding a big system in their heads, and agents do not get that context fatigue. This does not erase engineers, it moves the work. The ones scaling are good at directing agents, feeding them the right context, gating output with tests and guardrails, and reviewing like a senior to catch the confident wrong answer. Delegation became the new deep work. One story keeps getting passed around: Rakuten handed an AI coding agent a single task inside a large open-source project, walked away, and came back hours later to working code. It is a good story. It is mostly true. And the part everyone repeats, the hours and the accuracy number, is the least interesting part of it. The interesting part is why it worked at all, and what it says about the job of engineering now. This is an engineering perspective, not a vendor pitch. We direct coding agents on real client work every week, so this is less a hot take than a description of how the work actually changed. What actually happened at Rakuten? Here are the facts, stated carefully, because the viral version inflates one of them. At its Claude Opus 4 launch in May 2025, Anthropic said Rakuten had validated the model with a demanding open-source refactor that ran independently for about seven hours. The detail came later, in Rakuten's own customer story: an engineer pointed Claude Code at vLLM, the open-source inference engine, and asked it to implement a specific activation vector extraction method. The result reportedly hit 99.9% numerical accuracy against the reference implementation. Two honest corrections, because we would rather be right than dramatic: It was not unattended. The engineer described providing occasional guidance across those hours. \"Ran independently for seven hours\" is real and impressive. \"Fully autonomous, zero humans\" is not what happened. 99.9% is a narrow metric. It is the numerical accuracy of one method's output against a reference, not a general statement that the agent is 99.9% correct at everything. Useful, specific, and easy to misread as a headline. And the number the internet loves most, that vLLM is \"12.5 million lines of code,\" is the one to be most skeptical of. It appears in Rakuten's own copy, so people quote it in good faith, but it is off by roughly twenty times. The vLLM core is on the order of 84,000 lines of Python, and under 600,000 lines across all languages. You only reach 12.5 million by counting things nobody reasons about: full git history, every related repository, vendored and generated kernels. The real point survives the correction cleanly: Claude navigated a large, multi-language codebase it had never seen, in Python, CUDA, and C++, and landed a change in the right place. That is the impressive thing. It was never the line count. Why was context the real bottleneck, not intelligence? Strip the demo down and the lesson is almost boring. A senior engineer who already knew vLLM could have written that method too. A new hire could have, eventually, after weeks of wrong turns. The difference between the senior and the new hire was never raw intelligence. It was how much of the codebase each one could hold in their head and reason about at once. That is the bottleneck in most software work. Not \"can a smart person figure this out,\" but \"can anyone hold enough of this system in working memory to make the right change in the right place without breaking three others.\" On a large codebase that is genuinely hard, and it is hard in a way that has nothing to do with how clever you are. What does \"AI doesn't get context fatigue\" actually mean? Humans are bad at holding large context for long stretches, and not because we are dim. We tire. We forget the thing we read forty files ago. We take a break and lose the thread. We make small mistakes late in a long session that we would never make in the first hour. Holding a sprawling mental model of a system is exhausting, and exhaustion is where the bugs come from. A coding agent does not get tired in that way. It can keep the relevant slice of a large codebase in front of it and reason about it consistently at hour seven the same way it did at minute ten. That is the actual unlock in the Rakuten story. Not that the model is smarter than a senior engineer, but that it does not degrade across a long, context-heavy task the way a person does. Sustained context, not", "articleSection": "Engineering", "author": { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/" }, "dateModified": "2026-07-07", "datePublished": "2026-06-23", "description": "Rakuten pointed a coding agent at vLLM and had it implement a specific method, running about seven hours with occasional human guidance and hitting 99.9% numerical accuracy against the reference. The viral \"12.5 million lines of code\" figure is roughly twenty times too high; the real point is that the agent navigated a large, unfamiliar, multi-language codebase and landed the change in the right place. The lesson is that the bottleneck was never intelligence, it was context: humans tire and make mistakes holding a big system in their heads, and agents do not get that context fatigue. This does not erase engineers, it moves the work. The ones scaling are good at directing agents, feeding them the right context, gating output with tests and guardrails, and reviewing like a senior to catch the confident wrong answer. Delegation became the new deep work.", "headline": "The Bottleneck Was Never Intelligence. It Was Context.", "image": "https://wavect.io/img/blog/headers/header_ai-coding-agents-context-not-intelligence.svg", "inLanguage": "en", "mainEntityOfPage": { "@id": "https://wavect.io/blog/ai-coding-agents-context-not-intelligence/", "@type": "WebPage" }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/blog/ai-coding-agents-context-not-intelligence/", "wordCount": 2082 } ``` ```json { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "item": "https://wavect.io/", "name": "Home", "position": 1 }, { "@type": "ListItem", "item": "https://wavect.io/blog/overview/", "name": "Blog overview", "position": 2 }, { "@type": "ListItem", "item": "https://wavect.io/blog/topics/ai-agents/", "name": "AI and agents", "position": 3 }, { "@type": "ListItem", "item": "https://wavect.io/blog/clusters/agent-engineering/", "name": "Agent engineering", "position": 4 }, { "@type": "ListItem", "item": "https://wavect.io/blog/ai-coding-agents-context-not-intelligence/", "name": "AI Coding Agents 2026: Context Was the Bottleneck | ", "position": 5 } ] } ```