AI Agent SLA Template: Accuracy, Latency, Human Handoff and Auditability

An AI agent service-level agreement is a contract schedule that defines what a successful task is, how it is measured, when a human must take over, what evidence the supplier must retain, and what happens when the agent misses the agreed bar. Uptime alone is not enough. An API can return HTTP 200 while the agent cites a source that does not exist, calls the wrong tool, repeats a payment, or leaves a customer waiting in a dead queue.

This template targets the contract stage after a buyer has selected a workflow and a likely supplier. If you are still defining the build, use the AI MVP scope template. If you are deciding whether a pilot deserves production, use the AI pilot kill-or-scale scorecard. If you are still testing a vendor's security and data claims, use the EU AI vendor security questionnaire. This SLA starts where those artifacts stop: recurring production obligations, evidence and remedies.

This is a technical and commercial drafting aid, not legal advice. The bracketed values and sample targets are negotiation starting points. Adapt them to the workflow, risk classification, governing law, sector rules and liability structure with qualified counsel.

The copyable AI agent SLA schedule

Paste this into the service schedule, statement of work or order form. Fill every bracket. If a field has no owner, clock, denominator or evidence source, it is not yet enforceable.

Measurement rules that prevent an SLA dispute

The formula is intentionally simple: Successful Task Completion Rate = successful Eligible Tasks / all Eligible Tasks × 100. Do not use completed agent runs as the denominator. That removes timeouts, dead loops and failed handoffs, which are the failures the buyer is paying to control.

1. Define successful task completion as a business outcome

"The model returned an answer" is not completion. For a support agent, success might mean the correct policy answer was grounded in an approved source, the CRM was updated once, and the customer either received a correct resolution or entered the correct human queue. For an invoice agent, success might mean every required field matched the source document, the confidence rule was applied, and no payment was released without the named approval.

Write one completion predicate per task class. A safe refusal can count as correct behaviour but should not count as a completed business outcome unless the commercial model deliberately defines it that way. Report straight-through completion, safe refusal and human handoff separately so a supplier cannot hit the target by escalating everything.

2. Make material hallucination a severity definition, not a vibe

NIST's Generative AI Profile uses the term confabulation for confidently presented false or erroneous content. A contract needs a narrower test for materiality. The definition above asks two questions: is the assertion or action unsupported or false, and can it change an outcome or create meaningful harm?

A typo, harmless style defect or correctly labelled uncertainty is not a Material Hallucination. A fabricated refund policy, invented legal citation, nonexistent customer consent, wrong bank detail, or tool call built on made-up state is. A false draft caught by a required verifier before any user or external system sees it is a quality defect, but it is not an uncontained Severity 1 event. Preserve both counts. Containment should reduce severity, not erase evidence.

3. Measure tool use at first attempt and final outcome

Ultimate tool success alone rewards retry storms. First-attempt success alone ignores whether a transient error recovered safely. Report both, and keep every attempt in the trace. A successful call needs five things: the right authorised tool, valid arguments, permission at execution time, a verified response, and the intended state change exactly once.

Require idempotency keys for write actions such as refunds, bookings, emails and database updates. An unauthorised tool call, a duplicate irreversible action or a call that bypasses a mandatory approval is automatically unsuccessful and should trigger the agreed severity rule even if the downstream API returned success.

4. Put the whole workflow inside the latency clock

Measure the experience the buyer receives, not the fastest model span. The clock includes routing, retrieval, model calls, tool execution, validators, retries and supplier-controlled queue time. It stops only at completion, safe refusal, or a complete handoff. The human response clock then starts as a separate SLO.

Google's SRE guidance recommends percentile-based latency because averages hide the long tail. Use p50 for the typical case, p95 for the operating commitment and p99 where a plausible worst case matters. Set separate targets for interactive, voice, batch and high-risk approval tasks. They are different products from a latency perspective.

5. Contract the escalation path and the context bundle

Mandatory triggers should be observable rules, not the model saying it feels uncertain. Typical triggers include an out-of-scope request, missing required source, failed write tool after the retry budget, conflicting customer records, policy threshold, regulated decision, user request for a human, suspected prompt injection, or any action above an approval value.

A handoff succeeds only when it enters the named queue with enough context for a person to act without replaying the whole conversation. Measure trigger detection, routing accuracy, bundle completeness, queue acknowledgement, resolution and reopen rate. The ICO's human-review guidance also recommends logging human overrides and their reasons, which turns handoff data into evidence for the next eval-set revision.

6. Allocate unavailable dependencies instead of excluding them

List every material dependency: model API, vector store, identity provider, CRM, payment service, customer data source and human queue. For each one, name who selected and controls it, the timeout, retry budget, fallback, data-preservation behaviour and incident owner.

A customer-controlled outage can be excluded from the supplier's quality credit for the evidenced period, but it should still appear in dependency-availability and degraded-mode reporting. A model provider selected by the supplier should not become a blanket escape clause. The supplier may not control the outage, but it does control timeouts, circuit breakers, queueing, fallback, duplicate prevention and honest user messaging.

7. Freeze the eval set without freezing learning

Version the cases, expected outcomes, rubrics, graders, category weights and sampling method. Store a hash with every release. Neither party should delete a hard case, relabel a failure or change a grader solely to make the current system pass.

New risks still need to enter the set. The practical rule is dual reporting: run the frozen baseline and the proposed new set for one full measurement window, show the category deltas, approve the change, then make the new set authoritative from a stated date. Keep a hidden holdout and check for test contamination. NIST AI RMF calls for documented test sets, metrics and tools, evaluation under deployment-like conditions, and production monitoring. The contract should name the artifacts that prove those activities happened.

8. Treat model-provider changes as production releases

A material change includes the provider, model family, snapshot, routing rule, system prompt, tool schema, retrieval corpus, safety policy or grader. Pin versions where the provider supports it. OpenAI's own API compatibility guidance warns that prompting behaviour can change between model snapshots and recommends pinned versions plus evals.

Require notice, regression evidence by task segment, security review where the data path changes, cost and latency deltas, an approval owner and rollback. For a forced provider retirement, the supplier should present the migration evidence early enough to choose between approving the successor, routing to a fallback or suspending the affected task class.

9. Retain traces that prove the metric without retaining everything

Each task trace should include a unique task and trace ID, task class, timestamps, actor and tenant, model and prompt-template versions, retrieval document IDs and revisions, tool name and redacted arguments, result status, retries, approvals, handoff events, final disposition, metric reason codes and linked incident. Record raw prompt or output content only where the data schedule permits it.

OpenTelemetry's GenAI observability guidance shows agent, model and tool spans, and explicitly notes that recorded messages can contain sensitive data. Contract access control, redaction, regional storage, legal hold, export and deletion alongside retention. "We have logs" is not an audit clause if the customer cannot retrieve the evidence during a dispute.

10. Link severity to containment and commercial remedies

These are editable starting points. Response times should match service hours, risk and the supplier's support price.

Credits should be automatic from the shared evidence, not dependent on a short claim window that only the supplier can calculate. They also need an escalation path. A 10% credit does not make an unauthorised payment acceptable. For regulated financial entities, DORA Article 30 is a useful benchmark for the contract mechanics: precise quantitative and qualitative service levels, notice of material developments, ongoing monitoring, audit rights, corrective action and exit planning. DORA does not apply to every buyer, but the mechanics are sound.

Why the model provider's SLA does not cover your agent

The upstream SLA is one input to your architecture, not the customer outcome. For example, the Vertex AI platform SLA defines downtime through server-side error rate and offers tiered credits. It does not promise that an agent chose the right tool, cited a real source or completed a business process. Your supplier contract must cascade provider availability into an end-to-end design and add the missing quality, handoff and audit commitments.

This is also why price belongs beside reliability. A fallback that meets the SLO at ten times the normal cost may protect continuity and still destroy the business case. Track the commercial denominator with the AI agent cost per successful action model, then state who pays for retries, fallback providers, human rework and incident remediation.

EU procurement and compliance context

Do not claim that this template makes a system compliant. Classification and obligations depend on the intended purpose and sector. Where the EU AI Act's high-risk rules apply, the official text addresses automatic logs, human oversight, accuracy, robustness and cybersecurity in Articles 12, 14 and 15. The European public-buyers community also publishes model AI contractual clauses, while warning that they are not a complete agreement and must be customised.

The commercial SLA should sit beside, not replace, the data processing agreement, security schedule, IP terms, AI-use restrictions, regulatory allocation, insurance, liability and exit assistance. The ICO's AI contracts and third-parties guidance specifically recommends deciding acceptable accuracy before procurement and putting accuracy KPIs or SLAs into written supplier contracts.

"The hard part is not choosing 98% instead of 97%. It is defining the denominator, preserving the failed traces, and agreeing what the supplier must do when the agent is confidently wrong. That is where a metric becomes a contract."

Research sources

Researched and checked on 19 July 2026. Standards, provider terms and regulatory implementation can change, so verify the current source before signing.

1. NIST AI 600-1, Generative Artificial Intelligence Profile, confabulation and generative-AI risk controls.
2. NIST AI RMF Core, documented test sets, deployment-like evaluation and production monitoring.
3. Regulation (EU) 2024/1689, EU AI Act, especially Articles 12, 14 and 15 where high-risk obligations apply.
4. Regulation (EU) 2022/2554, DORA, Article 30 contract mechanics for applicable financial entities.
5. ICO, Contracts and third parties in AI, accuracy due diligence, written KPIs and audit clauses.
6. Google SRE Book, Service Level Objectives, indicators, percentiles, measurement windows and error budgets.
7. Google Cloud Vertex AI Platform SLA, an example of provider uptime, exclusions and credit mechanics.
8. OpenTelemetry, GenAI observability, agent, model and tool traces plus sensitive-content considerations.
9. OpenAI API backward compatibility guidance, model snapshots, pinning and evals.
10. AgentSLA: Towards a Service Level Agreement for AI Agents, research framing agent quality-of-service as an emerging specification problem.