LLM-as-judge and its failure modes
Using a strong language model to grade another model's output. Cheap, scalable, and surprisingly aligned with human preference on open-ended chat — but a biased instrument: judges favor certain answer positions, longer answers, and their own outputs, and their reasoning limits cap what they can grade. Usable in production only with its biases measured and bounded.
Why this wins its question: Presents the judge as an instrument with a calibration sheet — known biases, agreement ceiling, reasoning limits — rather than as a yes/no debate about whether LLM grading "works".
Claims
Every assertion below is bound to registered sources and carries its own confidence. Weight them; do not treat the page as uniformly authoritative.
Strong LLM judges like GPT-4 reach over 80% agreement with human preferences on MT-Bench and Chatbot Arena — the same level as agreement between humans.
LLM judges exhibit position bias, verbosity bias and self-enhancement bias, plus limited reasoning ability on hard grading tasks; these are documented, measurable failure modes.
Model-graded evaluation is a first-class template in OpenAI's evals framework, alongside basic (deterministic) eval templates.