Existing tools can't separate AI mistakes from real critique in peer reviews.

Scientists use artificial intelligence tools called Large Language Models to help review scientific papers. However, these tools sometimes hallucinate, meaning they make up smooth-sounding statements that the papers do not actually support. Checking these claims is tough because human reviewers must check facts across long, technical articles. To tackle this, researchers built a benchmark called HalluPeer. They created a system that matches paper text with human-written reviews and reviews intentionally altered with fake claims. An automated pipeline applies a custom set of error categories and inserts these flaws. When tested, existing detection tools struggled to tell real critiques apart from hallucinations. Furthermore, tests on authentic human reviews proved that these exact hallucination patterns already happen in real-world peer reviews.

Why it matters

Peer review decides what gets published in science, so if AI helpers invent facts, it undermines the entire assessment process. This matters for researchers, editors, and anyone relying on published science to be accurate.

Questions answered