Abstract
Results of Testing the Gold Standard 2-Week Reviewer Deadline
Emilie Gunn,1 Kelly Brooks,1 Stephanie Valladao1
Objective
Prior research has shown the effects of shortening a review deadline on overall time to review.1 We wanted to determine whether extending the standard 2-week deadline for peer review in a scholarly journal would have positive outcomes. In particular, we wanted to test the hypothesis that offering a 21-day deadline would increase the reviewer conversion rate, thus reducing the number of reviewers invited before getting 2 reviewers to agree. A secondary end point was the number of days it took reviewers to complete their reviews.
Design
In this pilot study, 593 review invitations for 95 manuscripts were sent from JCO Oncology Practice between July 1, 2024, and September 30, 2024, with a 21-day deadline. This was compared with 606 invitations for 116 manuscripts sent between July 1, 2023, and September 30, 2023, with a 14-day deadline. Only Original Reports and Reviews were studied because these article types require at least 2 reviewers. Using the Editorial Manager submission system, editors invited reviewers using the normal process, except that reviewers were offered 21 days instead of the industry-standard 14 to complete their reviews. Reviewers received information regarding the 21-day time frame at initial invitation, at the time of review invitation acceptance, and via reminders 1 week and 2 days prior to the deadline. The goal for each paper was to obtain 2 reviewers.
Results
When the time allowed to submit a review was increased to 21 days, the number of reviewers invited per manuscript increased by 0.5 reviewers (10.4%). The time it took reviewers to accept a review invitation increased by 0.1 days (12.5%), measured as the number of days from the date the invitation was sent to the date the reviewer agreed
(Table 25-0847). The percentage of late reviews decreased by 5% (from 27% to 22%). The median (IQR) time to submit a review was 19 (11-33) days, compared with 13 (5-18) days during the control period. Additionally, late reviews were submitted an average of 11.1 days late, a 29.1% increase. However, median (IQR) time from submission to first decision decreased by 4.7% to 41 (31-55) days from 43 (30-65.5) days.
Conclusions
Extending the 2-week deadline for peer review did not decrease the number of reviewers invited before getting 2 to agree. When the allowed time was increased, the time taken to review increased as well. Although there were fewer late reviews overall, the reviews submitted past the deadline were later than those submitted under the 14-day deadline. We ultimately recommended that the journal continue with the current 2-week deadline for reviewers. Further research is needed in this area.
Reference
1. Cochran A, Guertin L. Affecting editor behavior with data: a case study. Sci Editor. 2014;37(1):117-119.
1American Society of Clinical Oncology, Alexandria, VA, US, emilie. gunn@asco.org.
Conflict of Interest Disclosures
None reported.
Acknowledgment
We thank Jeffrey Peppercorn for his help with designing this abstract and Scott Marchese for his statistical expertise.