Original title: Statistical Methods for Assessing Non-Replicable, Outlying, and Influential Studies
Yefeng Yang, Shinichi Nakagawa · September 2026
Researchers often combine results from multiple experiments using a method called meta-analysis. However, unusual studies can distort the overall conclusion. In practice, researchers struggle to spot and understand these odd results. The authors address this issue. First, they explain that non-replicable studies, statistical outliers, and influential studies are distinct ideas, not interchangeable terms. Next, they review diagnostic procedures to help scientists detect these problematic studies and explain the statistical thinking behind them. They also describe modern improvements, like methods for handling correlated or imprecise sampling errors, robust checks, and graphical tools that make unusual patterns easier to see. Finally, they provide practical recommendations so scientists can carefully interpret studies that seem non-replicable, outlying, or unusually influential.
Imagine you are grading a group project where each student wrote a section. One student's section is very different in style. You need to decide if it is just poorly written, if it contradicts the others, or if it is so good that it changes the whole grade. In this paper, the group project is a meta-analysis, and the student sections are individual studies. The authors discuss how to tell apart a merely unusual study from one that actually undermines the whole conclusion.
When researchers combine multiple studies to reach a conclusion, a single unusual study can distort the overall result. This paper helps distinguish whether that study is just statistically extreme, genuinely non-replicable, or overly influential. Understanding the difference is crucial for making trustworthy evidence-based decisions in medicine, social science, and policy.
The paper is a review and does not introduce new statistical methods or tests on real data. Its recommendations are qualitative and may require expert judgment to apply. The abstract does not provide empirical validation of the discussed procedures.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Statistical Methods for Assessing Non-Replicable, Outlying, and Influential Studies — Yefeng Yang, Shinichi Nakagawa — Mathematics — doi:10.3390/math14193436
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.