Abstract
Efficiency of Author Anonymization in Peer Review
Markus K Heinemann,1 Andreas Boening,2 Kazunori Okabe,3 Jessica Bogensberger,4 Zulfugar Timur Taghiyev2
Objective
Double-anonymized peer review is thought to enhance objectivity but causes additional work for editorial staff.1-3 The aim of this study was to evaluate its efficiency in a midsized cardiothoracic surgical journal.
Design
In July 2017, the journal introduced anonymization, performed by a secretary during the first editorial office check, and introduced custom questions to the review form asking reviewers if they had an idea about the origin of a manuscript despite its anonymization and, if so, why. Descriptive statistics were used for analysis, and a 2-proportion z test was utilized as applicable.
Results
Between July 2017 and November 2024, a total of 1735 manuscripts were amenable for anonymization and sent for peer review to a mean of 2 reviewers. For 525 manuscripts (30.3%), at least 1 reviewer had an idea about its origin for at least 1 of 4 potential reasons (multiple entries possible; n = 597): faulty anonymization (232), references (172), familiar with work (146), and characteristic language (47). In 82 manuscripts (15.6%), more than 1 reason was given by at least 1 reviewer. Forty guesses (7.6%) were incorrect for the following reasons: misled by impressions of being familiar with the work (20), references (9), faulty anonymization (6), and characteristic language (5). Multiple reviewers guessed for the same reason in 8 cases and for different reasons in 32 cases. One of the reviewers was incorrect in 5 manuscripts; all reviewers guessed correctly in 35. Among 649 accepted manuscripts (37.4% of the total manuscripts), 223 (34.3%) were deanonymized by reviewers, whereas 426 stayed anonymized (65.6%). The 2-proportion z test demonstrated higher acceptance rates for anonymized compared with deanonymized manuscripts (426 vs 223; z = −11.27).
Conclusions
Perfect anonymization remains challenging, if not impossible, because hints identifying origins can be hidden throughout a manuscript. Roughly 30% of manuscripts were correctly deanonymized by the reviewers of an admittedly limited specialty surgical community. Even though the results seem to support different editorial dispositions of anonymized vs deanonymized manuscripts, editors must be aware of the confined reliability of anonymization and weigh it against the additional administrative workload.
References
1. Parmanne P, Laajava J, Järvinen N, Harju T, Marttunen M, Saloheimo P. Peer reviewers’ willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review. Res Integr Peer Rev. 2023;8:14. doi:10.1186/s41073-023-00140-6
2. Ucci MA, D’Antonio F, Berghella V. Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. Am J Obstet Gynecol MFM. 2022;4(4):100645. doi:10.1016/j.ajogmf.2022.100645
3. Kmietowicz, Z. Double blind peer reviews are fairer and more objective, say academics. BMJ. 2008;336(7638):241. doi:10.1136/bmj.39476.357280.DB
1German Society for Thoracic and Cardiovascular Surgery (DGTHG), Germany, heinemann@uni-mainz.de; 2Universitaetsklinik Giessen, Germany; 3Bell Land General Hospital, Osaka, Japan; 4Thieme Publishers, Stuttgart, Germany.
Conflict of Interest Disclosures
Jessica Bogensberger is an employee of Thieme Publishers, the publishing house of the journal investigated.