Original title: LLM measurements that agree with human coders overall can still distort specific effects
Alexander K. Moore, Bingqing Li, Shane Wang, Matt Thomson · September 2026
Scientists often use artificial intelligence models to score data instead of human workers. To test if an AI does a good job, researchers usually check if its overall scores match human scores. But this general check hides serious problems. In a study using AI to rate pictures, the researchers found that even when an AI matched humans well overall, it still gave misleading results for specific questions. The AI exaggerated differences, shrank them, or flipped them backward. It also favored certain categories over others. Changing the instructions given to the AI helped some comparisons but made others worse. Because these errors repeat across instructions, simply mixing prompts will not fix them. Instead, researchers must test AI on specific claims.
Imagine a teacher grading essays by how well they match a sample answer on average. One student's essay gets a B overall, matching the sample's tone and length. But the teacher misses that the student's argument about climate change is completely backward. In this paper, the LLM is like that teacher: it matches human ratings on average but can systematically distort the specific comparisons researchers care about, like which category of food is rated higher.
Social scientists are starting to use LLMs to replace human coders for tasks like rating photos or text. This paper shows that even when an LLM's average ratings match humans, it can systematically distort the very effects researchers are trying to measure—inflating, shrinking, or even reversing them. That means studies relying on LLM coders could reach wrong conclusions without any obvious warning sign.
The abstract does not specify which open-weight LLMs were tested, the exact prompts used, or the size of the biases relative to human disagreement. It also does not establish how often such distortions occur across different domains or tasks beyond food and scene photographs.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
LLM measurements that agree with human coders overall can still distort specific effects — Alexander K. Moore, Bingqing Li, Shane Wang et al. — SocArXiv — doi:10.31235/osf.io/dz3qe_v1
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.