Agent Reliability

comparison 2

• Deterministic harnesses vs LLM-as-judge: when each wins

  A decision comparison between mechanical grading (state diffs, executable tests) and model-based grading. Deterministic harnesses win wherever an outcome can be computed; judges win on open-ended quality — at the cost of documented biases that must be measured before the judge's verdicts mean anything.

• Public benchmarks vs private task suites: what each buys you

  Public benchmarks buy comparability, transparency and community scrutiny; private suites buy contamination resistance, domain fit and Goodhart resistance. Production teams need both, in a fixed shape: public suites to place yourself in the field, a private rotating suite to gate your releases. Using either for the other's job is how teams fool themselves.

entity 25

• Observability standards for agents (OpenTelemetry GenAI)

  The emerging standard layer for agent telemetry: OpenTelemetry's GenAI semantic conventions define spans, metrics and events for model calls, tool execution, MCP interactions and agent operations, so traces from different stacks become comparable. Still in active development — adopt it for vocabulary and interoperability, pin versions, and expect churn until stabilization.

• Release gates for agent systems

  The ship/no-ship decision expressed as declared, machine-checkable thresholds instead of judgment calls: minimum external quality score, minimum factuality, minimum interface success rate, zero critical security findings. Declaring the gate in configuration makes it auditable and Goodhart-visible; a gate that lives in someone's head renegotiates itself under deadline pressure.

• Agent telemetry and actor classification

  Instrumenting a deployed agent so every action is attributable: which actor (human, autonomous agent, crawler, operator) initiated the request, which tools were called, what state changed. Actor classification is the prerequisite for everything downstream — rate limits, audits, incident forensics, and knowing whether a regression hit humans or machines.

• AgentBench

  A multi-environment benchmark evaluating LLMs as agents across eight distinct settings (operating system, database, knowledge graph, games, web tasks and more). Its durable findings: commercial frontier models act competently as agents while sub-70B open models lag sharply, and the failures concentrate in long-horizon reasoning, decision-making and instruction following — not in single-turn knowledge.

• Behavioral canaries

  Small, fixed probe tasks run continuously against a deployed agent. Each canary has a known-good outcome; a change in that outcome is treated as an early regression signal — model drift, prompt rot, tool API changes — before users feel it. The chaos-engineering steady-state hypothesis applied to agent behavior instead of infrastructure metrics.

• Check-counter regression alarm

  A meta-guardian that watches the guardians: the validator exports the number of distinct check families it runs, and CI tracks that counter. If a refactor silently drops a check, the count falls and the alarm fires — catching the failure mode where protection disappears while every remaining light stays green.

• Deterministic vs probabilistic evaluation

  Two grading regimes for agent behavior. Deterministic evaluation checks outcomes mechanically — exact state comparison, executable tests — and reproduces run to run. Probabilistic evaluation delegates judgment to a model or human grader, trading reproducibility for coverage of open-ended outputs. Production reliability work starts deterministic and adds probabilistic grading only where mechanical checks cannot reach.

• Evaluating retrieval (RAG) inside agents

  The retrieval layer of an agent fails in three distinct ways — wrong passages retrieved, right passages used unfaithfully, fluent answer ungrounded in either — and each needs its own metric. Reference-free RAG evaluation makes this measurable without annotated ground truth, at the price of inheriting LLM-judge biases. In an agent, retrieved content is also an attack surface, so RAG evaluation and injection testing share fixtures.

• Fault injection for agents

  Deliberately breaking the world around an agent to observe how it fails: tool errors, malformed API responses, timeouts, permission denials, adversarial content in retrieved data. The agent equivalent of chaos engineering's real-world events — the goal is verified recovery behavior, not the absence of faults.

• GAIA benchmark

  A benchmark of 466 real-world questions for general AI assistants, designed so answers are unambiguous to grade but require reasoning, multi-modality, web browsing and tool use to reach. Its signature result is the human-AI gap: 92% for human respondents against 15% for GPT-4 with plugins at publication — questions conceptually simple for people, hard for tool-using models.

• Goodhart resistance in agent metrics

  Designing agent evaluation so that optimizing the reported number does not destroy what the number means. Known countermeasures: separate the metric you optimize from the metric you report, measure consistency across repeated trials rather than best-of-N, and audit with a panel of honesty figures instead of a single score.

• Guardian falsification

  A discipline for proving that an automated check (a "guardian") can actually fail: deliberately plant a defect, watch the guardian go red, restore the correct state, watch it go green. A guardian that has never been observed red is an assumption, not a control. Practiced in production in the Citarium framework, where every validator check family traces back to a real incident.

• Human approval gates and graduated autonomy

  The control that keeps an agent's blast radius human-sized: specific action classes (destructive, irreversible, outward-facing) require a human yes before execution, and autonomy expands only as the agent earns it with evidence. The complement to sandboxing — the sandbox contains what the agent can do, gates contain what it may do.

• Independent third-party evaluation

  Evaluation performed by organizations that did not build the system: external red teams, research nonprofits running autonomy evaluations, and government institutes publishing open harnesses. Self-evaluation has a structural conflict of interest — the same incentive gradient Goodhart describes — and the emerging ecosystem of independent evaluators is the field's answer. For buyers, a vendor's willingness to be independently evaluated is itself a reliability signal.

• Inspect (UK AI Security Institute evaluation framework)

  The open-source evaluation framework from the UK AI Security Institute: composable solvers and scorers, sandboxed execution of untrusted model code, agent primitives, and a registry of 200+ prebuilt evals. The closest thing agent evaluation has to shared infrastructure — METR's Hawk platform builds on it — and the default answer to "should we write our own harness runner?" (usually no).

• LLM-as-judge and its failure modes

  Using a strong language model to grade another model's output. Cheap, scalable, and surprisingly aligned with human preference on open-ended chat — but a biased instrument: judges favor certain answer positions, longer answers, and their own outputs, and their reasoning limits cap what they can grade. Usable in production only with its biases measured and bounded.

• Opaque rotating test sets

  Held-out evaluation tasks that the evaluated system (and its training pipeline) never sees, refreshed on a schedule. Opacity prevents direct optimization against the benchmark; rotation limits contamination and slow overfitting as public data leaks into training corpora. The price is comparability: rotated scores are trends, not fixed leaderboards.

• OSWorld

  A benchmark of 369 open-ended tasks executed in real operating systems (Ubuntu, Windows, macOS) spanning web and desktop apps, file I/O and multi-application workflows. Every task ships its own initial-state setup and an execution-based evaluation script, making it a working template for reproducible computer-use agent evaluation. Headline gap at publication: humans 72.36%, best model 12.24%.

• Process vs outcome evaluation of agent trajectories

  Two places to point the grader: the final state (outcome) or every intermediate step (process). Outcome grading is cheap, objective and gameable by lucky trajectories; step-level grading catches reasoning that arrives at the right answer wrong, at the cost of expensive labels. The research result: process supervision beat outcome supervision decisively on hard math. For agents, outcome grading gates releases while process signals explain failures and catch right-for-the-wrong-reason passes.

• Prompt-injection testing for agents

  Systematically probing whether an agent acts on instructions smuggled into content it reads — tool results, retrieved documents, web pages, emails. For tool-using agents this is the top-ranked risk turned into a test surface: every untrusted input channel gets seeded with adversarial instructions, and the assertion is that the agent's actions never follow them. Runs in the sandbox, gates the release.

• Sandboxed execution environments

  Contained environments where an agent can act — run code, call tools, mutate state — without its failures reaching production systems. Execution-graded benchmarks already presume one; for dangerous-capability testing the sandbox is the safety case. A sandbox that leaks (network egress, shared state, real credentials) converts a test failure into an incident.

• Statistical rigor in agent evals

  Treating an eval score as a measurement with uncertainty instead of a fact: report error bars, model your task set as a sample from a larger population of possible tasks, and never call a two-point difference a regression (or an improvement) without knowing the noise floor. The cheapest upgrade available to most eval pipelines is a standard error next to every number.

• SWE-bench Verified

  The human-validated subset of SWE-bench: 500 tasks filtered from the original test set for quality, graded by unit-test verification against post-PR behavior. Its existence is the interesting fact — a flagship benchmark needed a human audit of its own tasks, which makes "who benchmarks the benchmark?" a standing question for anyone gating decisions on eval numbers.

• Testing MCP servers and integrations

  What reliability testing means for Model Context Protocol integrations: conformance to the JSON-RPC contract, capability-negotiation edge cases, and — because the spec itself declares tool descriptions untrusted — injection probes through every server-supplied string. An MCP interface is an API whose consumer is a model, so its test suite gates on success rate like any other API, plus adversarial content like no other API.

• WebArena

  A self-hosted, realistic web environment for evaluating autonomous agents: functional sites for e-commerce, forum discussion, collaborative software development and content management, plus maps and knowledge-base tools. Tasks are long-horizon and graded on functional correctness of the end state. Headline result at publication: best GPT-4 agent 14.41% against human 78.24%.

faq 1

• Agent reliability FAQ

  The questions teams actually ask before shipping an autonomous agent: how to stop hallucinated actions against critical APIs, what a benchmark should measure, whether one successful demo means anything, and where to start when there is no eval at all. Answers are bound to sources, not opinion.

glossary 1

• Agent reliability glossary

  Working definitions for the terms this instance uses everywhere: eval, harness, canary, red-teaming, regression gate, pass^k, fault injection, sandbox. Each definition is bound to a source that uses the term operationally, so the glossary states how the field works rather than how a dictionary reads.

guide 9

• Calibrating your LLM judge before it gates anything

  The protocol that turns an LLM judge from vibes into an instrument: measure its agreement against a human-audited sample, probe its known biases directly (position, verbosity, self-preference), put error bars on every comparison it feeds, and define the triggers that force recalibration. A judge is promoted to gatekeeper only after this protocol — and demoted the day its judge model changes.

• Choosing k and the threshold for pass^k

  The corpus recommends pass^k everywhere and never says which k or what number to demand. This is the arithmetic and the decision rule: k comes from how many times the task runs between human reviews, the threshold comes from the cost of one failure, and both must be set before you see the score. Includes why pass@k and pass^k move in opposite directions and why measured pass^k is almost never p^k.

• Designing an agent sandbox that actually contains failures

  Requirements and checks for an execution environment where an agent can fail safely: no default egress, ephemeral seeded state, fake credentials that look real, a bounded blast radius, and an observation channel the agent cannot write to. Includes a containment checklist you can test the same way you test the agent.

• How to falsify your first guardian

  A step-by-step procedure for proving that one automated check in your agent pipeline can actually fail: pick the guardian, plant a defect it must catch, confirm red, restore, confirm green, and record the episode. Thirty minutes of work that converts a green light from an assumption into evidence.

• From incident to check: making failures pay rent

  The loop that turns every production incident into a permanent guardian: write the incident down, derive the smallest check that would have caught it, falsify that check, and add it to a counted inventory so it can never silently vanish. After enough cycles, your validator is a compressed history of everything that has actually gone wrong.

• Measuring agent reliability: bounded metrics and decision tables

  Which numbers actually predict whether an agent can ship: consistency across repeated trials (pass^k), execution-verified success rates, task time horizon, and a panel of honesty figures instead of one score. Includes a decision table mapping each question a team asks to the metric that answers it.

• Running your first agent red-team exercise

  A bounded, repeatable adversarial exercise for a tool-using agent: scope the rules of engagement inside the sandbox, enumerate the attack surfaces (injection channels, tool permissions, data paths), run manual and automated passes, triage by blast radius, and convert every confirmed finding into a falsified check so the exercise compounds instead of evaporating.

• Regression-gating a model upgrade

  The playbook for swapping the model behind a production agent without discovering the differences in production: freeze the harness, baseline the incumbent with pass^k on critical paths, run the candidate on the identical suite, diff the panel, roll out behind canaries. Model swaps are not optional — providers retire models on notice — so this gate runs on the provider's calendar, not yours.

• Your first agent evals: from zero to a release gate in two weeks

  The bootstrap path for a team whose agent has no evals at all: harvest real tasks from production logs, write execution-based graders for them, adopt an existing eval framework instead of building one, baseline with repeated trials and error bars, and wire the result into CI as a gate. Two weeks of part-time work; the output is the suite every other reliability practice hangs from.

Machine surfaces

Everything a human can read here, an agent can read as data. Each link below is a real file served from this origin.

• /llms.txt llms-index
• /llms-full.txt llms-full
• /sitemap.xml sitemap
• /robots.txt robots
• /feed.json feed-json
• /feed.xml feed-rss
• /graph.json graph
• /api/index.json entity-index
• /.well-known/mcp/server.json mcp-manifest
• /skill.json skill-manifest
• /skill/SKILL.txt skill
• /skill/SKILL.md skill-alias

Agents can also query this corpus over MCP — see the agent interface.