Abstract
Association Between Number of External Peer Review Invites, Unsuccessful Invites, and Declined Reviews With Rejection of Manuscripts
Gene Y-K Ong,1,2,3 Ellen Weber,3,4 Joshua McAlpine5
Objective
Many journals encounter difficulties obtaining peer reviewers. However, it is unknown if this is associated with publication outcomes.1,2 The purpose of this research was to evaluate whether, for initial manuscript submissions that were eventually peer reviewed, there was a significant association between the number of external peer reviewers sought, unsuccessful invites, and declined invites for external peer review and the publication outcome.
Design
This was a retrospective study of anonymized, unique, original research submissions to the Emergency Medicine Journal (EMJ) that received at least 1 external peer review over a 5-year period (January 2016 to December 2020). A database of deidentified original manuscripts submitted to EMJ during the study period was interrogated to determine if there were significant associations between the number of unsuccessful external peer review invitations, the number of total invitations needed, and the acceptance or rejection of a research manuscript. Original submissions without any external peer review were excluded. Statistical review invitations were excluded from the data. Mann-Whitney U test was used to assess differences between the variables and their publication outcomes. Odds ratios (OR), likelihood ratios (LR), and positive predictive value (PPV) were used as measures of association for potential thresholds for variables and publication outcomes.
Results
There were 806 deidentified peer-reviewed original submissions included with 85 manuscripts (10.6%) accepted for publication during the study period (Table 79). The ORs for a peer-reviewed original research submission eventually being rejected, according to number of invitations, were 77.5 (95% CI, 24.2-248.2; P < .001; (LR, 21.0; 95% CI, 6.9-63.7; PPV, 99.4%; 95% CI, 98.3-99.8) for submissions with 4 or more invitations, 36.8 (95% CI, 9.0-150.9; P < .001; LR, 20.0; 95% CI, 5.1-78.8; PPV, 99.4%; 95% CI, 97.7-99.9) for submissions with 3 or more unsuccessful external peer review invitations, and 22.6 (95% CI, 3.1-163.8; LR, 18.0; 95% CI, 2.6-127.2; PPV, 99.4%; 95% CI, 95.6-99.9) for submissions with 2 or more peer reviewers who declined review invitations.
Conclusions
The number of declined peer review invitations and total review invitations prior to a decision were associated with rejection of a manuscript. The wide 95% CIs in these results could be due to the high variability of underlying factors that could have influenced the difficulty in getting peer reviews and their interplay with the decision to publish. The findings of this study may also be potentially different for different journals.3 Further research should be done to provide further insights on specific factors that may be associated with difficulties in getting peer reviewers.1-3
References
1. Tvina A, Spellecy R, Palatnik A. Bias in the peer review process: can we do better? Obstet Gynecol. 2019;133(6):1081-1083. doi:10.1097/AOG.0000000000003260
2. Tennant JP, Dugan JM, Graziotin D, et al. A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Res. 2017;6:1151. doi:10.12688/f1000research.12037.3
3. Stamm T, Meyer U, Wiesmann HP, Kleinheinz J, Cehreli M, Cehreli ZC. A retrospective analysis of submissions, acceptance rate, open peer review operations, and prepublication bias of the multidisciplinary open access journal. Head Face Med. 2007;3:27. doi:10.1186/1746-160X-3-27
1Department of Emergency Medicine, KK Women’s and Children’s Hospital, Singapore, geneong@yahoo.com; 2Duke-National University of Singapore Medical School, Singapore; 3Emergency Medical Journal, London, UK; 4Department of Emergency Medicine, University of California, San Francisco, San Francisco, CA, USA; 5BMJ Publishing Group, London, UK
Conflict of Interest Disclosures
None reported.