Original title: Clean Engineering, Unstable Measurement: A Preregistered Reliability Failure of Black-Box LLM Observers on Shared Endpoints
Haoyaun Zhu, Jie Zhang · September 2026
Many developers use AI models as judges to score text, filter data, and decide rankings. This relies on an unstated assumption: asking the same model name the exact same question tomorrow will give the same answer. Researchers tested this across 52,988 request attempts. They sent identical evaluation prompts into shared AI systems to see if the scores stayed steady. They failed completely. Rankings sent within the same window agreed poorly, and exact replays sent the next day also disagreed. Three things caused this: the labels used to assign meaning biased the outputs, true differences between options were far smaller than the system's background noise, and identical inputs produced differing rankings. Waiting days, switching between four providers, or tweaking metrics did not fix it. Running models locally helped only when servers were quiet. The authors warn that online model names are not stable instruments.
Imagine you ask a friend to grade the same essay twice on different days. One time they give it an A, another time a B, even though the essay didn't change. The problem is that your friend's judgment shifts unpredictably, so the grades aren't reliable for ranking or decisions. Here, the AI judge behaves like that friend, giving different scores for identical inputs because its internal process isn't stable.
Language-model judges decide what data trains AI and which systems win leaderboards. If they give inconsistent scores, those decisions are built on shaky ground, affecting everyone who relies on fair AI evaluation.
The abstract states that all results are based on externally measured behavior on shared serving infrastructure, so findings may not apply to privately hosted or fully controlled LLM systems.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Clean Engineering, Unstable Measurement: A Preregistered Reliability Failure of Black-Box LLM Observers on Shared Endpoints — Haoyaun Zhu, Jie Zhang
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.