Engaging doctoral students in peer review: a pre-post study evaluating the effectiveness of the 'Peerspectives' course on review quality, knowledge and skills

ObjectiveTo assess the effectiveness of the Peerspectives course on editor-judged quality of peer review DesignSingle arm, pre-post interventional study SettingPeerspectives is a peer review training course developed for doctoral students at the Charite - Universitatsmedizin Berlin, Germany ParticipantsDoctoral students in health research domains who enrolled in the Peerspectives course between October 2020 and August 2022 and consented to participate (N = 80). Intervention: A semester-long course (approx. 18 weeks) providing training about the structure, purpose, and conduct of peer review and editing processes in biomedical journals. The course consisted of 12 hours of interactive lectures, weekly homework assignments, and 12 hours of hands-on, small-group workshops under guidance of mentors with reviewing and editing experience. Students peer-reviewed real scientific manuscripts under review at a partnering journal, The BMJ. Main outcome measuresThe overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, participants self-assessed knowledge and skills related to scholarly peer review (1-5 Likert scale). ResultsAfter course completion, participants RQI scores were higher than before the course (median increase of 0.5 points, p<0.001; mean increase of 0.36 points, p<0.001). The RQI scores of participants post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Largest improvements were seen in understanding what is expected from a reviewer (increase in means from 2.9 to 4.5), confidence in own ability to review (2.5 to 3.9), and knowing what to look for in a manuscript while reviewing (2.8 to 4.2). ConclusionsProviding doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback. Study pre-registration: https://osf.io/vndcx