A Pilot Program for Early Career Mentorship in Journal Peer Review

Anjali Garg,¹ Preeti Panda,² Lydia Furman,³ Kimberly Montez,⁴ Alex R. Kemper,⁵ Lewis First⁶

Objective

To implement and evaluate a pilot mentorship program in which experienced peer reviewers mentor novice reviewers in writing peer reviews.

Design

In 2023, editorial board members of Pediatrics, the peer-reviewed journal of the American Academy of Pediatrics, volunteered for a pilot mentorship program to improve the peer review skills of early career physicians (medical students to junior faculty) to enhance the number of early career reviews within the pool present at Pediatrics. Mentors selected 1 or 2 mentees. The target participation for the pilot was 10 dyads. Literature review and expert consensus informed development of peer review mentorship materials. Curricula for mentees included didactics and checklists, and material for mentors provided feedback guidance. For objective assessment, the review quality instrument (RQI) was adapted into an 8-item, 3-point Likert scale questionnaire to score mentee reviews.¹ Mentees needed to complete at least 3 reviews with a score of 3 on at least 6 of 8 RQI items to graduate the program—a metric selected by the program leaders. Mentors and mentees completed a review of a regular article (as defined by the Pediatrics author guidelines) separately, then met together for structured feedback from the mentor to the mentee on the review. Each regular article was sent to the mentor based on their area of expertise; the turnaround time for completion was 3 weeks. The RQI was completed by the mentor, and the combined review was ultimately submitted to the editorial board by the mentor. Qualitative data regarding skills gained and areas for improvement were compiled and analyzed from intake and exit surveys completed by mentors and mentees.

Results

Eleven dyads enrolled, with 9 graduating from the program, an 81% graduation rate. Five dyads (55%) required 3 reviews to graduate, 3 (33%) required 4 reviews, and 1 (9%) required 5 reviews. The median number of RQI items scoring a 3 in each dyad’s first reviews increased from 4 (IQR, 2-7) to 7 (IQR, 6-8) by the last review. The range of total RQI scores increased from 16-24 (first reviews) to 23-24 (last reviews) for each dyad. Qualitative analysis indicated that the pilot program was feasible and effective. Notably, mentees reported increased comfort with peer review and improvement in their own manuscript writing skills. Mentors noted an improvement in their own reviewing skills through mentorship. Areas for improvement included targeted communication from the leadership team and allowing separate enrollment of mentees and mentors with matching by the program team to further enhance dissemination of this program.

Conclusions

The pilot program was successful in enrolling the target number of participants, graduating over 80% of the mentees with improvement of reviewing skills per increase in RQI scores, and demonstrated feasibility for future program building.

Reference

1. van Rooyen S, Black N, Godlee F. Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. J Clin Epidemiol. 1999;52(7):625-629. doi:10.1016/s0895-4356(99)00047-5

¹Division of Critical Care Medicine, Department of Pediatrics, Ann and Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL, US, anjaliga07@gmail.com; ²Division of Pediatrics, Department of Emergency Medicine, Stanford University, Stanford, CA, US; ³University Hospitals Rainbow Babies & Children’s, Cleveland, OH, US; ⁴Department of Pediatrics, Wake Forest University School of Medicine, Winston-Salem, NC, US; ⁵Division of Primary Care Pediatrics, Nationwide Children’s Hospital, Columbus, OH, US; ⁶Department of Pediatrics, University of Vermont Children’s Hospital, Burlington, VT, US.

Conflict of Interest Disclosures

All authors are members of the Pediatrics editorial board.

Acknowledgment

We acknowledge Kate Larson and the Pediatrics editorial board administrative team for their help with the implementation of this program.

Additional Information

Preeti Panda is a co–corresponding author (preetipanda@stanford.edu).