Abstract
Implementation of a Peer Reviewer Probation and Performance Monitoring Program at The Journal of Bone & Joint Surgery
Marc Swiontkowski,1 Christina Nelson1
Objective
The Journal of Bone & Joint Surgery (JBJS) has been in publication for more than 125 years. Because of its relatively high Impact Factor and reputation, the JBJS reviewer roster is large and subject to steady growth. However, similar to other publications, JBJS is plagued with uneven reviewer performance. In recent years, several measures have been enacted to refine the reviewer database. Our goal was to evaluate the initial effect of these measures.
Design
To encourage better reviewer performance, the editorial department contacted all individuals who met any 1 of a variety of poor-performance indicators in January 2016 (N = 612). These indicators included declining at least 3 consecutive review invitations, a less than 50% review-completion rate, not completing any reviews within the past 3 years, or low review grades in 2015. Each reviewer was informed of his or her removal from the reviewer roster, and those who requested to remain as a reviewer were flagged as “on probation” in the manuscript tracking system. Editors also had an opportunity to retain specific reviewers in the database, and these reviewers were also placed on probation. The performance of each probationary reviewer was then monitored closely over the following year with the goal of improving performance and consequently removing the probationary designation. A stagnant or poorer performance after 1 year would lead to the removal of that reviewer from the roster without notification.
Results
The data for the 126 reviewers placed on probation are shown in the Table. This group comprised approximately 9% of the overall reviewer database. There was a more than 50% reduction in the number of declined reviews, with an overall average of 2.12 declined invitations per reviewer before the probation period began and an average of 0.91 declined invitations per reviewer during the year following program implementation. There was also a reduction in the number of reviewers who were uninvited owing to no invitation response. The quality of the reviews was unchanged, and the mean number of days to acceptance of an invitation was not statistically different. There was also a decrease in the number of days to complete the review.
Conclusions
A reviewer probation program appears to be successful in improving reviewer performance in an established surgical journal. In tandem with this program, we hope that the introduction of additional resources and incentives such as the Elite Reviewers Program and individual thank-you emails to reviewers who provide high-quality reviews will further enhance overall reviewer performance at JBJS.
1The Journal of Bone & Joint Surgery, Needham, MA, USA, cnelson@jbjs.org
Conflict of Interest Disclosures:
None reported.