How random is the review outcome? A systematic study of the impact of external factors on eLife peer review
Liang, W.; Mahowald, K.; Raymond, J.; Krishna, V.; Smith, D.; Jurafsky, D.; McFarland, D.; Zou, J.
Show abstract
The advance of science rests on a robust peer review process. However whether or not a paper is accepted can depend on random external factors--e.g. the timing of the submission, the matching of editors and reviewers--that are beyond the quality of the work. This article systematically investigates the impact of these random factors independent of the papers quality on peer review outcomes in a major biomedical journal, eLife. We analyzed all of the submissions to eLife between 2016 to 2018, with 23,190 total submissions. We examined the effects of random factors at each decision point in the review process, from the gatekeeping senior editors who may desk-reject papers to review editors and reviewers who recommend the final outcome. Our results suggest that the peer-review process in eLife is robust overall and that random external factors have relatively little quantifiable bias.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Researcher and Clinician Preferences for a Journal Transparency Tool: A Mixed-Methods Survey and Focus Group Study 90%
- Experts fail to reliably detect AI-generated histological data 90%
- Full Publication of Preprint Articles in Prevention Research: An Analysis of Publication Proportions and Results Consistency 90%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.