Abstract
Peer Reviewers’ Willingness to Review and Their Recommendations After the Finnish Medical Journal Changed From Single-Anonymous to Double-Anonymous Peer Review
Piitu Parmanne,1 Joonas Laajava,2 Noora Järvinen,3 Terttu Harju,4,5,6 Mauri Marttunen,7,8 Pertti Saloheimo3
Objective
Peer reviewers’ willingness to review, their recommendations, and quality of reviews were explored after switching from single-anonymous to double-anonymous peer review in the Finnish Medical Journal, published in Finnish. Previous research has mainly concerned journals published in English.1
Design
The Finnish Medical Journal switched to double-anonymous peer review on September 1, 2017. The material comprised reviews submitted from September 2017 to February 2018. The controls were the reviews submitted between September 2015 and February 2016 and between September 2016 and February 2017. The reviews on all manuscripts with at least 2 reviews were included. In cases with more than 2 reviews for the manuscript, the first 2 reviews received were included. The number of invitations needed to receive 2 reviews for a manuscript was calculated. How often the reviewers recommended to accept as is, minor revision, major revision, or to reject was also explored. The contents of the reviews were independently assessed by 2 experienced reviewers (T.H., M.M.) who were unaware of the peer review model and the decisions made on the manuscripts. The Review Quality Instrument2 was modified to apply for both original research and review manuscripts. The reviewers’ recommendations were tested using the χ² test, and the means of quality assessments were tested with an independent-samples t test. A P value <.05 was considered statistically significant.
Results
A total of 118 reviews given for 59 original research and review manuscripts by 114 individual double-anonymous reviewers were included. These were compared with 232 reviews for 116 manuscripts by 213 reviewers who were single-anonymous, ie, the reviewers were aware of the review process. The number of invitations needed to obtain 2 reviews was similar during the double-anonymous and single-anonymous periods: median, 3 (IQR, 2-4 reviews; range, 2-7 reviews). When performing a double-anonymous review, the reviewers more seldom recommended accept as is and minor revision than during the control period (accept as is, 8.5% vs 12.1%; minor revision, 50.8% vs 60.8%, respectively), and more often reviewers chose major revision and reject (major revision, 33.1% vs 22.8%; reject, 7.6% vs 4.3%, respectively; P = .07). For the quality assessment, 116 reviews were included (in 2 cases there was no written review) and compared with 104 reviews given between September 2016 and February 2017. The results are shown in Table 72.
Conclusions
In general, the quality of double-anonymous reviews was significantly better than that of single-anonymous reviews, contrary to a previous meta-analysis.1 Switching to double-anonymous review did not alter the reviewer’s willingness to review. The reviewers became slightly more critical and more often recommended major revision or reject, but this study was underpowered to show statistical significance. The study period was limited by the change in peer review platform in March 2018. Another limitation was that there was no control for the quality of the manuscripts.
References
1. Bruce R, Chauvin A, Trinquart L, Ravaud P, Boutron I. Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC Med. 2016;14(1):85. doi:10.1186/s12916-016-0631-5
2. van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. BMJ. 1999;318(7175):23-27. doi:10.1136/bmj.318.7175.23
1Finnish Medical Association, Helsinki, Finland; 2University of Helsinki, Helsinki, Finland; 3Finnish Medical Journal, Helsinki, Finland, pertti.saloheimo@laakarilehti.fi; 4Respiratory Medicine Research Unit, Department of Medicine, University of Oulu, Oulu, Finland; 5Medical Research Center, Oulu University Hospital, Oulu, Finland; 6OYS Somatics, Internal Medicine Centre, Pulmonary Outpatient Clinic, Oulu University Hospital, Oulu, Finland; 7Department of Adolescent Psychiatry, Helsinki University Hospital and University of Helsinki, Helsinki, Finland; 8Department of Public Health Solutions, Finnish Institute for Health and Welfare in Finland, Helsinki, Finland
Conflict of Interest Disclosures
None reported.
Funding/Support
The study was partially funded by a grant from the Finnish Association for Scholarly Publishing.
Role of the Funder/Sponsor
The funder had no impact on the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.
Acknowledgment
We thank designer Lauri Parikka for technical assistance in collecting the data.