Original title: On the Cost of Conforming to Reviewers’ Expectations and the Potential Benefit of Prediction Competitions
Ido Erev, Adi Tarabeih, Rachel Barkan · September 2026
Peer reviewExperts evaluate research before publication. helps science, but it can also trap it. When reviewers share rigid assumptions, scientists feel pressured to follow along instead of exploring bolder ideas. The authors propose a fix: add open prediction competitions to peer review. In these contests, different computer models compete to forecast real-world data. To show how this works, the authors looked at two competitions predicting risky choices. For more than 70 years, standard theories assumed people trust descriptions of rewards, judge options alone, and weigh odds in their heads. However, the winning models won using opposite ideas: they assumed people doubt stated descriptions, recall similar past memories, and reuse what worked before. Running prediction contests lets fresh, better ideas beat entrenched assumptions through actual predictive success.
Imagine a group of friends deciding where to eat. They all have the same favorite restaurant, so they always go there, even though other places might be better. In this paper, the friends represent researchers who all share the same assumptions, and the restaurant is the standard model. The prediction competition is like trying different restaurants and picking the best one based on actual taste tests.
When reviewers all share the same assumptions, researchers stick to those ideas even if better ones exist. This slows scientific progress. Prediction competitions offer a way to test new ideas directly, encouraging creativity and grounding evaluation in what actually predicts outcomes.
The paper only demonstrates the idea with two competitions in one domain (decisions under risk). It does not test whether this approach works in other fields or whether journals would adopt it. The competitions themselves may have limitations, such as requiring resources to organize. The abstract does not discuss potential downsides of competitions, like gaming or overfitting to past data.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
On the Cost of Conforming to Reviewers’ Expectations and the Potential Benefit of Prediction Competitions — Ido Erev, Adi Tarabeih, Rachel Barkan — Entropy — doi:10.3390/e28101044
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.