How to falsify your first guardian
A step-by-step procedure for proving that one automated check in your agent pipeline can actually fail: pick the guardian, plant a defect it must catch, confirm red, restore, confirm green, and record the episode. Thirty minutes of work that converts a green light from an assumption into evidence.
Why this wins its question: A reproducible procedure with a live production example (the Citarium validator) behind every step — not testing philosophy. The reader can run the whole cycle against their own CI in one sitting.
Key takeaways
- A guardian that has never been observed red is an assumption, not a control.
- The cycle is plant a defect, expect red, restore, expect green — then record the episode.
- Red for the wrong reason falsifies a different check; the target stays unproven.
- Falsify every new guardian in the pull request that lands it, and re-falsify after pipeline refactors.
Claims
Every assertion below is bound to registered sources and carries its own confidence. Weight them; do not treat the page as uniformly authoritative.
The falsification cycle is plant a defect, observe the guardian go red, restore the correct state, observe it go green; a guardian is only trusted after it has been seen in both states.
In the Citarium framework the content validator is the first guardian falsified: it runs from CI on day one and each of its check families exists because an incident paid for it.
Robust evaluations are extremely difficult to develop and implement, which is why an untested check deserves no trust by default.
Why falsify
A guardian — any automated check standing between your agent and production — earns trust the same way a scientific hypothesis does: by surviving an attempt to break it. A check that has always been green tells you nothing; maybe it works, maybe it silently stopped running, maybe it never could fail. Robust evaluations are hard to build (claim c3), so the default assumption about any untested check is that it is broken.
The cycle (claim c1)
1. Pick one guardian. Start with the check whose failure would hurt most: schema validation, a claim-to-source binding check, a destructive-action gate in your agent harness. 2. Plant a defect. Introduce the exact class of error the guardian exists to catch. Make it minimal and realistic: an unregistered source id, a future date in a freshness label, a tool call outside the allowlist. Commit it on a branch, never on main. 3. Expect red. Run the pipeline. The guardian must fail, and fail for the planted reason — read the error message and verify it points at your defect, not at an accidental one. 4. Restore. Revert the planted defect completely. 5. Expect green. Run the pipeline again. It must pass. If it stays red, your restore was incomplete or the guardian has a state leak — both are findings. 6. Record the episode. Note the guardian, the defect class, the observed red, and the date. This record is what "our CI catches X" now cites.
Worked example
The Citarium content validator ships as each instance's first guardian (claim c2). Falsifying it on this very repository looks like: add a claim citing a source id that is not in `sources.yaml`, run the test command, observe the cross-reference check fail with the exact claim id, revert, observe green. The validator's own source code documents that every one of its check families traces to a real incident — falsification keeps that inventory honest.
After the first one
- Falsify each new guardian when it lands, in the same pull request.
- Re-falsify after any refactor of the pipeline that runs the guardians.
- Track the number of active checks so a silently dropped guardian is
itself an alarm — see the check-counter regression alarm.
Failure modes to expect
- The guardian passes with the defect planted. The check does not
cover what you thought. Fix the check before trusting anything else.
- Red, but for the wrong reason. Your defect tripped a different
check. You have falsified that one instead; the target is still unproven.
- Green after restore fails. Cached state or ordering dependence in
the pipeline. Treat as a real defect in the harness.