Abstract
Editor Initial Manuscript Review:
A Masked Pilot Study
Douglas K. Novins,1 Mary K. Billingsley,2 Robert R. Althoff3
Objective
For many journals in science and medicine, a large number of submitted manuscripts may be rejected by editors before peer review. Given the high rates of such “desk rejections,” it is important that editorial reviews minimize bias in this decision-making process. One potential source of bias is the editor having access to information regarding the authors of manuscripts, which may result in editorial decision being influenced by authors’ academic reputation as well as other author characteristics.1,2 Unfortunately, a recent review was unable to identify any empirical studies of such “triple-blind” peer review.3 In this pilot study, we masked author information in the initial stage of editorial review to explore whether this changes patterns in the articles that our journal sends out for peer review.
Design
The Journal of the American Academy of Child and Adolescent Psychiatry utilizes a 2-round editor in chief (EIC) review process during which the EIC is aware of author identity in both rounds of review. In the standard round 1 review, the EIC completes a high-level reading of the manuscript and the cover letter, checks for test recycling, and reviews study preregistration. In the standard round 2 review, the EIC completes a detailed read of the manuscript. In this preregistered pretest/posttest study (https://doi.org/ 10.17605/OSF.IO/TNYFX), we followed the above standard process in the pretest. In the posttest, the identity of the authors was masked in the round 1 review and tasks that unmask author identity, such as reading the cover leader, were moved to the round 2 review.
Results
A total of 95 manuscripts were included in the pretest and 102 were included in the posttest. In the masked process, significantly more manuscript decisions were made in round 2 (49/102) than in the unmasked process (26/95) (χ² = 8.91; 1 degree of freedom; P < .01) (Figure 25-1112). Overall, EIC decisions (reject, transfer, assignment to an action editor) did not differ between the pretest and posttest (χ² = 2.48; 2 degrees of freedom; P = .29).
Conclusions
The shift of decisions from round 1 to round 2 in the posttest review process is consistent with the shift of key EIC tasks to the round 2 review, resulting in more manuscripts also receiving a detailed read of their manuscript than in the pretest.
References
1. Fox CW, Meyer J, Aimé E. Double-blind peer review affects reviewer ratings and editor decisions at an ecology journal. Funct Ecol. 2023;37(5):1144-1157. doi:10.1111/1365-2435.14259
2. Srivastava DS, Bernardino J, Marques AT, et al. Editors are biased too: an extension of Fox et al. (2023)’s analysis makes the case for triple-blind review. Funct Ecol. 2024;38(2):278-283. doi:10.1111/1365-2435.14483
3. Polnaszek BE, Mei J, Cheng C, et al. Triple-blind peer review in scientific publishing: a systematic review. Am J Obstet Gynecol MFM. 2024;6(4):101320. doi:10.1016/j.ajogmf.2024.101320
1Journal of the American Academy of Child and Adolescent Psychiatry (University of Colorado Anschutz Medical Campus); 2Journal of the American Academy of Child and Adolescent Psychiatry (American Academy of Child and Adolescent Psychiatry); 3Journal of the American Academy of Child and Adolescent Psychiatry (University of Vermont).
Conflict of Interest Disclosures
Douglas K. Novins reported receiving honorarium for serving as editor in chief for the Journal of the American Academy of Child and Adolescent Psychiatry, American Academy of Child and Adolescent Psychiatry and grant support from the National Institutes of Health. Robert R. Althoff reported receiving honorarium for serving as associate editor for the Journal of the American Academy of Child and Adolescent Psychiatry from the American Academy of Child and Adolescent Psychiatry; grant support from National Institutes of Health and Klingenstein Third Generation Foundation; and stock or Equity in WISER Systems, LLC.
Acknowledgment
We acknowledge the contributions of our journal’s senior editorial team in designing this study.