Original title: Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection
Joe Cecil, Marjorie Freedman · September 2026
Chatbots can sound confident while sharing wrong medical facts. Spotting these mistakes is hard because small errors often hide inside mostly accurate paragraphs. Usually, researchers have just one person check the text once, which misses a lot. To fix this, researchers tested a layered approach. First, human reviewers read the responses. Next, an artificial intelligence judge searched for extra suspicious claims. Finally, medical experts and evidence-based fact-checkers reviewed all the flagged issues to decide what was truly wrong. The researchers found that human reviewers missed real mistakes on their first pass. The artificial intelligence tool found more possible errors, but it also missed things humans noticed. Even the expert reviewers sometimes disagreed. Ultimately, relying on a single quick review misses mistakes, while multi-step checking catches far more errors.
You're proofreading a friend's essay and think it's correct after a fast read. But when you go back with a highlighter and ask a teacher to review it, hidden errors appear. In this paper, chatbot text is the essay, first-pass annotation is your quick read, and multi-perspective methods are the highlighter and teacher.
When medical chatbots give wrong information, people might make bad health decisions. Current detection methods miss many errors, so safer chatbots need better checking systems.
The study focuses on medical chatbot responses and may not apply to other fields. Adjudicator disagreement shows that results depend on the experts used. The method improves coverage but does not eliminate all errors or judgment biases.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection — Joe Cecil, Marjorie Freedman
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.