---
title: "Semaprax Context Benchmark: Token and Cost Evidence"
canonical: https://wavect.io/semaprax/benchmarks/
language: en
description: "What the Semaprax Agent Context benchmark measures today, what it cannot prove about LLM tokens or cost, and the protocol needed for credible comparisons."
image: "https://wavect.io/img/general/bak/open_graph_preview.jpg"
---

BENCHMARK · AGENT CONTEXT V1

# Context is measurable. Cost savings are not proven yet.

Publish the counterexample before the claim.

The current benchmark freezes a small source corpus and compares deterministic context artifacts. It is a compiler-mechanics benchmark, not a model or repository economics result.

semaprax://benchmarks v0.2

```
entity     Semaprax
status     pre-alpha research
verified   2026-08-11
authority  github.com/wavect/semaprax
```

## Is Semaprax cheaper for coding agents?

Not proven. Semaprax is designed to let agents request typed, bounded context, but the current Agent Context v1 benchmark does not measure model tokens, latency, answer quality, accepted-task rate, or repository-scale cost. On its small corpus, the context artifact is larger than the source.

// 01

## What the current benchmark measures

Frozen input

A small versioned Semaprax corpus and maintenance task

Deterministic output

Agent Context v1 JSON with explicit byte, node, and depth limits

Comparison boundary

Source bytes versus structured context bytes for the fixed corpus

// 02

## What it does not measure

- Model input or output tokens
- Wall-clock latency or provider price
- Answer correctness or accepted patches
- Repository-scale navigation and maintenance
- A comparison with Rust, C, C++, Go, or another language

// 03

## The evidence needed before a cost claim

1. **Freeze equivalent maintenance tasks and repository snapshots.**
2. **Run the same model, harness, tool permissions, and stopping rules.**
3. **Measure billed tokens, elapsed time, accepted-task rate, and regression failures.**
4. **Publish raw traces, exclusions, unsuccessful runs, and confidence intervals.**
5. **Repeat across repository sizes and more than one model family.**

GitHub specifications are the normative source. This page is a dated research summary. [GitHub repository](https://github.com/wavect/semaprax).

// FAQ

## Questions, answered without the hype

### Does Semaprax use fewer tokens than Rust or C?

There is no credible evidence for that comparison yet. The current benchmark does not run a model or compare languages, and its small structured context artifact is larger than the source.

### Why publish a benchmark that does not show savings?

Because it fixes the measurement contract and exposes counterevidence before a marketing claim. That makes later model and repository-scale results auditable.

Research project by Wavect: [Wavect GmbH](/). Created by Wavect as an open-source systems research project.

## Structured Data

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@id": "https://wavect.io/#organization",
      "@type": [
        "Organization",
        "ProfessionalService",
        "LocalBusiness"
      ],
      "employee": [
        {
          "@id": "https://wavect.io/team/kevin-riedl/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Kevin Riedl",
          "url": "https://wavect.io/team/kevin-riedl/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        },
        {
          "@id": "https://wavect.io/team/christof-jori/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Christof Jori",
          "url": "https://wavect.io/team/christof-jori/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        }
      ],
      "founder": [
        {
          "@id": "https://wavect.io/team/kevin-riedl/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Kevin Riedl",
          "url": "https://wavect.io/team/kevin-riedl/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        },
        {
          "@id": "https://wavect.io/team/christof-jori/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Christof Jori",
          "url": "https://wavect.io/team/christof-jori/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        }
      ],
      "legalRepresentative": [
        {
          "@id": "https://wavect.io/team/kevin-riedl/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Kevin Riedl",
          "url": "https://wavect.io/team/kevin-riedl/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        },
        {
          "@id": "https://wavect.io/team/christof-jori/#person",
          "@type": "Person",
          "jobTitle": "Managing Director",
          "name": "Christof Jori",
          "url": "https://wavect.io/team/christof-jori/",
          "worksFor": {
            "@id": "https://wavect.io/#organization",
            "@type": [
              "Organization",
              "ProfessionalService",
              "LocalBusiness"
            ]
          }
        }
      ],
      "name": "Wavect GmbH",
      "subjectOf": {
        "@id": "https://wavect.io/verified-claims.json#dataset",
        "@type": "Dataset",
        "creator": {
          "@id": "https://wavect.io/#organization",
          "@type": [
            "Organization",
            "ProfessionalService",
            "LocalBusiness"
          ]
        },
        "description": "A machine-readable registry of quantitative and qualitative claims published by Wavect, with review dates, localized page appearances and public third-party citations where available.",
        "inLanguage": "en",
        "isAccessibleForFree": true,
        "license": "https://creativecommons.org/licenses/by/4.0/",
        "name": "Wavect verified publication claims",
        "url": "https://wavect.io/verified-claims.json"
      },
      "url": "https://wavect.io/"
    },
    {
      "@id": "https://wavect.io/team/kevin-riedl/#person",
      "@type": "Person",
      "jobTitle": "Managing Director",
      "name": "Kevin Riedl",
      "sameAs": [
        "https://www.wikidata.org/wiki/Q139796365",
        "https://www.linkedin.com/in/wsdt",
        "https://github.com/wsdt"
      ],
      "url": "https://wavect.io/team/kevin-riedl/",
      "worksFor": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      }
    },
    {
      "@id": "https://wavect.io/team/christof-jori/#person",
      "@type": "Person",
      "jobTitle": "Managing Director",
      "name": "Christof Jori",
      "sameAs": [
        "https://www.wikidata.org/wiki/Q139796367",
        "https://www.linkedin.com/in/jocr77/",
        "https://github.com/jo-chris"
      ],
      "url": "https://wavect.io/team/christof-jori/",
      "worksFor": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      }
    },
    {
      "@id": "https://wavect.io/#website",
      "@type": "WebSite",
      "inLanguage": [
        "en",
        "de",
        "es",
        "zh"
      ],
      "name": "Wavect",
      "potentialAction": {
        "@type": "SearchAction",
        "query-input": "required name=search_term_string",
        "target": {
          "@type": "EntryPoint",
          "urlTemplate": "https://wavect.io/search/?q={search_term_string}"
        }
      },
      "publisher": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "url": "https://wavect.io/"
    },
    {
      "@id": "https://wavect.io/semaprax/benchmarks/#webpage",
      "@type": "WebPage",
      "dateModified": "2026-08-11",
      "inLanguage": "en",
      "isPartOf": {
        "@id": "https://wavect.io/#website",
        "@type": "WebSite"
      },
      "lastReviewed": "2026-08-11",
      "url": "https://wavect.io/semaprax/benchmarks/"
    },
    {
      "@id": "https://wavect.io/verified-claims.json#semaprax-context-benchmark-nonclaim",
      "@type": "Claim",
      "appearance": {
        "@id": "https://wavect.io/semaprax/benchmarks/#webpage",
        "@type": "WebPage",
        "inLanguage": "en",
        "url": "https://wavect.io/semaprax/benchmarks/"
      },
      "author": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "citation": {
        "@type": "WebPage",
        "url": "https://github.com/wavect/semaprax/blob/main/docs/AGENT-ECONOMICS-V1.md"
      },
      "dateModified": "2026-08-11",
      "identifier": "semaprax-context-benchmark-nonclaim",
      "inLanguage": "en",
      "text": "The current Semaprax Agent Context v1 benchmark does not measure model tokens, latency, answer quality, accepted-task rate, or repository-scale cost, and its small-corpus context artifact is larger than the source."
    }
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@id": "https://wavect.io/semaprax/benchmarks/#webpage",
      "@type": "WebPage",
      "creator": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "dateModified": "2026-08-11",
      "description": "What the Semaprax Agent Context benchmark measures today, what it cannot prove about LLM tokens or cost, and the protocol needed for credible comparisons.",
      "inLanguage": "en",
      "mainEntity": {
        "@id": "https://wavect.io/semaprax/benchmarks/#article",
        "@type": "TechArticle"
      },
      "name": "Semaprax Context Benchmark: Token and Cost Evidence | Wavect",
      "url": "https://wavect.io/semaprax/benchmarks/"
    },
    {
      "@id": "https://wavect.io/semaprax/benchmarks/#article",
      "@type": "TechArticle",
      "about": {
        "@id": "https://wavect.io/semaprax/#software",
        "@type": "SoftwareSourceCode"
      },
      "author": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "dateModified": "2026-08-11",
      "datePublished": "2026-08-11",
      "description": "What the Semaprax Agent Context benchmark measures today, what it cannot prove about LLM tokens or cost, and the protocol needed for credible comparisons.",
      "headline": "Context is measurable. Cost savings are not proven yet.",
      "inLanguage": "en",
      "isPartOf": {
        "@id": "https://wavect.io/semaprax/#webpage",
        "@type": "WebPage"
      },
      "mainEntityOfPage": {
        "@id": "https://wavect.io/semaprax/benchmarks/#webpage"
      },
      "publisher": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "url": "https://wavect.io/semaprax/benchmarks/"
    },
    {
      "@id": "https://wavect.io/semaprax/#software",
      "@type": "SoftwareSourceCode"
    },
    {
      "@id": "https://wavect.io/semaprax/benchmarks/#faq",
      "@type": "FAQPage",
      "inLanguage": "en",
      "mainEntity": [
        {
          "@type": "Question",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "There is no credible evidence for that comparison yet. The current benchmark does not run a model or compare languages, and its small structured context artifact is larger than the source."
          },
          "name": "Does Semaprax use fewer tokens than Rust or C?"
        },
        {
          "@type": "Question",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Because it fixes the measurement contract and exposes counterevidence before a marketing claim. That makes later model and repository-scale results auditable."
          },
          "name": "Why publish a benchmark that does not show savings?"
        }
      ],
      "url": "https://wavect.io/semaprax/benchmarks/"
    },
    {
      "@id": "https://wavect.io/verified-claims.json#semaprax-context-benchmark-nonclaim",
      "@type": "Claim",
      "appearance": {
        "@id": "https://wavect.io/semaprax/benchmarks/#webpage",
        "@type": "WebPage",
        "inLanguage": "en",
        "url": "https://wavect.io/semaprax/benchmarks/"
      },
      "author": {
        "@id": "https://wavect.io/#organization",
        "@type": [
          "Organization",
          "ProfessionalService",
          "LocalBusiness"
        ]
      },
      "citation": {
        "@type": "WebPage",
        "url": "https://github.com/wavect/semaprax/blob/main/docs/AGENT-ECONOMICS-V1.md"
      },
      "dateModified": "2026-08-11",
      "identifier": "semaprax-context-benchmark-nonclaim",
      "inLanguage": "en",
      "text": "The current Semaprax Agent Context v1 benchmark does not measure model tokens, latency, answer quality, accepted-task rate, or repository-scale cost, and its small-corpus context artifact is larger than the source."
    }
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "item": "https://wavect.io/",
      "name": "Home",
      "position": 1
    },
    {
      "@type": "ListItem",
      "item": "https://wavect.io/semaprax/",
      "name": "Semaprax",
      "position": 2
    },
    {
      "@type": "ListItem",
      "item": "https://wavect.io/semaprax/benchmarks/",
      "name": "Context is measurable. Cost savings are not proven yet.",
      "position": 3
    }
  ]
}
```