Original title: Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review
Peer reviewA process in which experts assess research before a journal decides whether to publish it. is when experts assess research before a journal publishes it. This paper examines what might change if reviewers and editors use language models to help write their reports and decision letters. These tools may save time and help produce constructive feedback, but their training data and inner workings are not fully transparent. The authors recommend disclosing their use and keeping people responsible for accuracy, fairness, and confidential information.
Imagine asking a friend to help write feedback on someone’s school project. The help could save time, but you may not know where their suggestions came from, and they might repeat an unfair assumption. In this paper, that friend’s help stands for language models assisting reviewers and editors; the concern is that useful support can come with hidden bias and risks to private information.
Researchers depend on peer review to judge and improve their work, while editors use it to shape what gets published. Language models may help reviewers and editors write reports faster, but unclear data handling and possible bias could affect both the fairness and confidentiality of those decisions.
The paper offers a small-scale exploration of ChatGPT and discusses potential effects; it does not establish how LLM use changes peer review across journals or over time. The abstract also notes unresolved uncertainties about bias, confidentiality, reproducibility, and effects on academic communities.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review — Mohammad Hosseini, Serge P. J. M. Horbach — Research Integrity and Peer Review — doi:10.1186/s41073-023-00133-5
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.