Original title: Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation
Danting Zhang, Bei Peng, Robert Loftin · September 2026
Standard tests show that large language models are great at content moderation. But these tests usually group many different rules into a single pass-or-fail label. Because of this, we cannot tell if models truly understand each separate rule. Researchers built a tool called Diagnostic Evaluation of COntent, or DECO. It breaks content down independently of the rules so scientists can run controlled tests on individual criteria. They also compared model decisions across different rules for the very same input text. They discovered that high benchmark scores hide major failures. Models struggle when a decision requires checking a specific rule instead of just sensing general harmfulness. Good overall scores do not prove models can reliably follow individual moderation guidelines.
Imagine grading a student's essay on multiple criteria like grammar, structure, and clarity. If you only give an overall grade, you might miss that the student is great at grammar but terrible at structure. In this paper, content moderation benchmarks give an overall score, but DECO tests each criterion separately, revealing that models fail on specific aspects even when overall scores are high.
This matters because content moderation keeps online spaces safe. If models can't follow specific rules, harmful content might slip through, or harmless content might be wrongly removed.
The study is limited to four moderation datasets and four LLMs, so results may not apply to all models or contexts. It focuses on criterion-level evaluation but doesn't address other aspects of moderation like speed or scalability.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation — Danting Zhang, Bei Peng, Robert Loftin
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.