{
  "id": "from-incident-to-check",
  "type": "guide",
  "name": "From incident to check: making failures pay rent",
  "summary": "The loop that turns every production incident into a permanent guardian: write the incident down, derive the smallest check that would have caught it, falsify that check, and add it to a counted inventory so it can never silently vanish. After enough cycles, your validator is a compressed history of everything that has actually gone wrong.",
  "locale": "en",
  "tags": [
    "guide",
    "incidents",
    "guardians",
    "falsification",
    "process"
  ],
  "relations": [
    {
      "rel": "applies",
      "target": "guardian-falsification"
    },
    {
      "rel": "applies",
      "target": "check-counter-alarm"
    },
    {
      "rel": "related",
      "target": "falsify-your-first-guardian"
    }
  ],
  "questions": [
    "What should happen after an agent incident?",
    "How do I stop the same failure from happening twice?",
    "Where do good eval checks come from?"
  ],
  "claims": [
    {
      "id": "c1",
      "text": "The Citarium validator is built this way in production: its source states that every check exists because an incident paid for it, and individual checks carry their incident of origin in code comments (the locale-contract check cites the pt-BR incident directly).",
      "sources": [
        "citarium-core-validator"
      ],
      "confidence": 0.95
    },
    {
      "id": "c2",
      "text": "New checks enter service through falsification — plant the incident's defect, observe red, restore, observe green — so the check demonstrably catches the failure it was born from.",
      "sources": [
        "citarium-methodology-brief"
      ],
      "confidence": 0.9
    },
    {
      "id": "c3",
      "text": "The check inventory is itself guarded: the validator exports its count of check families for CI to track, so a check silently dropped in a refactor raises an alarm.",
      "sources": [
        "citarium-core-validator"
      ],
      "confidence": 0.95
    }
  ],
  "takeaways": [
    "Every incident produces exactly one durable artifact — a falsified, counted check.",
    "Smallest check that rejects the defect wins; broad checks cry wolf and get deleted.",
    "Write the incident of origin into the check so the next maintainer knows why it exists.",
    "Only the counter notices a check's absence — uncounted protection disappears silently."
  ],
  "faqs": [],
  "evidence_tier": "primary",
  "evidence": {
    "level": "production",
    "source_types": [
      "production_system",
      "personal_experience"
    ]
  },
  "moat_flag": true,
  "winning_edge": "The loop is documented from a validator that was actually built this way — check by check, incident by incident — with the code as evidence. Postmortem literature stops at \"write action items\"; this guide's action item is always the same executable artifact.",
  "confidence": 0.9,
  "last_verified": "2026-08-08",
  "body": "## The rule\n\nEvery incident produces exactly one durable artifact: a check that\nwould have caught it, proven by falsification, added to a counted\ninventory. Everything else in the postmortem — narrative, blame,\nslideware — decays. The check compounds.\n\n## The loop, step by step\n\n1. **Capture the defect while it is fresh.** Not the outage story — the\n   minimal wrong state. An agent cited an unregistered source; a label\n   promised a locale the prose broke; a date claimed the future. One\n   sentence, one reproducible state.\n2. **Derive the smallest check that rejects that state.** Smallest\n   matters: a check that rejects too much gets deleted the first time\n   it cries wolf, and then the incident can return. Precision over\n   recall is the survival trait for guardians (the Citarium\n   locale-contract check is deliberately high-precision for exactly\n   this reason — claim c1).\n3. **Falsify it before trusting it.** Plant the original incident's\n   defect, watch the new check go red, restore, watch green (claim c2).\n   The incident becomes the check's own test case — the one defect this\n   check must provably catch forever.\n4. **Count it.** Add the check to the tracked inventory so its\n   disappearance is an alarm, not a mystery discovered during the next\n   incident (claim c3).\n5. **Write the origin into the check.** A comment naming the incident\n   turns the validator into documentation: the next maintainer learns\n   why the check exists before deciding to weaken it.\n\n## What this produces over time\n\nA validator built by this loop is a compressed incident history (claim\nc1): reading its checks is reading everything that has gone wrong,\nwith each entry executable and self-verifying. Teams that skip the\nloop keep relearning failures; teams that run it convert failure into\ninfrastructure at a fixed exchange rate — one incident, one guardian.\n\n## Anti-patterns\n\n- **The broad check.** Born from an incident but rejecting a whole\n  category \"to be safe\". It fires on legitimate work, gets deleted,\n  and takes the incident's protection with it.\n- **The uncounted check.** Added in a hurry, never inventoried,\n  silently lost in a refactor. Falsified once is not falsified\n  forever — only the counter notices absence.\n- **The narrative postmortem.** Action items assigned, none of them a\n  check. Six months later the document exists and the protection does\n  not.",
  "canonical_url": "https://agentreliability.dev/k/from-incident-to-check",
  "api_url": "https://agentreliability.dev/api/k/from-incident-to-check.json",
  "jsonld": {
    "@context": "https://schema.org",
    "name": "From incident to check: making failures pay rent",
    "description": "The loop that turns every production incident into a permanent guardian: write the incident down, derive the smallest check that would have caught it, falsify that check, and add it to a counted inventory so it can never silently vanish. After enough cycles, your validator is a compressed history of everything that has actually gone wrong.",
    "url": "https://agentreliability.dev/k/from-incident-to-check",
    "license": "https://spdx.org/licenses/CC-BY-4.0.html",
    "dateModified": "2026-08-08",
    "citation": [
      {
        "@type": "CreativeWork",
        "name": "Citarium content validator source (@citarium/core v0.1.0, compiler/validate.ts)"
      },
      {
        "@type": "CreativeWork",
        "name": "agent-reliability editorial brief and blueprint (Gate 1 approved, 2026-08-08)"
      }
    ],
    "author": {
      "@type": "Person",
      "name": "Santiago Santa María Morales",
      "jobTitle": "practitioner — harness engineering and agent evaluation in production"
    },
    "@type": "Article",
    "headline": "From incident to check: making failures pay rent"
  }
}