Consistency and Completeness of Retractions in Public Health Research on COVID-19
Abstract
Caitlin J. Bakker,1,2 Erin E. Reardon,3 Sarah Jane Brown,4 Nicole Theis-Mahon,4 Sara Schroter,5,6 Lex Bouter,7,8 Maurice P. Zeegers2
Objective
The scientific community responded rapidly to COVID-19, producing over 200,000 publications in 1 year.1 This speed brought challenges, including a higher retraction rate.2 While retraction helps correct the scientific record, retracted status may be inconsistently presented and notices may be incomplete.3 During a pandemic, inconsistency and incompleteness can have immediate and long-term health impacts. We evaluated retraction presentation consistency and notice completeness for COVID-19 vs non–COVID-19 publications. We describe reasons for retraction and time from publication to retraction.
Design
In March 2023, we retrieved retracted publications categorized as research articles or clinical studies in the subject area of public health and safety from Retraction Watch. A previous study focused on all retracted publications in this period3; this is a sub-study comparing COVID-19 with non–COVID-19 publications. Between April 28 and June 6, 2023, we assessed consistency in 11 databases (Academia.edu, CINAHL, Embase.com, Ovid Embase, Ovid Medline, PubMed, ResearchGate, SciHub, Scopus, Web of Science, and publisher websites) using 12 criteria from the International Committee of Medical Journal Editors and National Library of Medicine and notice completeness using 17 criteria from Retraction Watch and the Committee on Publication Ethics. Criteria were scored 0 if missing or 1 if present; partial scores were assigned when evaluating multicomponent criteria, such as bidirectional links. To ensure consistent scoring, a random subset of 21% (92 of 441) of retracted publications were independently reviewed by 2 researchers. Each researcher extracted data using Qualtrics forms, and scoring discrepancies were resolved by consensus. Following this calibration phase, remaining publications were extracted by a single reviewer. Kruskal-Wallis tests assessed differences in scores between COVID-19 and non–COVID-19 publications.
Results
Of 441 publications, 47 were about COVID-19 and 394 were not. COVID-19 publications were published between 2019 and 2022, while non-COVID 19 publications were published between 1978 and 2022. COVID-19 publications were most frequently retracted due to concerns about reliability of data or results (15 [31.9%]) compared with plagiarism (82 [20.8%]) for non–COVID-19 publications. The median time between publication and retraction was 120 (IQR, 15-196) days for COVID-19 publications and 326 (IQR, 124-789) days for non–COVID-19 publications (P < .001). Across 11 databases, 41.2% (110 of 267) of records retrieved for retracted COVID-19 publications were marked as retracted compared with 47.6% (1225 of 2574) for non–COVID-19 publications. There was no statistically significant difference between consistency or completeness scores for COVID-19 vs non–COVID-19 retracted publications (Table 25-1062). No publications met all criteria.
Conclusions
Incomplete and inconsistent information poses challenges for researchers and practitioners, undermining trust in scientific literature. We found no association between publications being about COVID-19 and the consistency or completeness of retraction information; however, publications about COVID-19 appeared to be retracted more quickly.
References
1. Shimray SR. Research done wrong: a comprehensive investigation of retracted publications in COVID-19. Account Res. 2022;30(7):393-406. doi:10.1080/08989621.2021.2014327
2. Yeo-Teh NSL, Tang BL. An alarming retraction rate for scientific publications on coronavirus disease 2019 (COVID-19). Account Res. 2020;28(1):47-53. doi:10.1080/08989621.2020.1782203
3. Bakker CJ, Reardon EE, Brown SJ, et al. Identification of retracted publications and completeness of retraction notices in public health. J Clin Epidemiol. 2024;173:111427. doi:10.1016/j.jclinepi.2024.111427
1University of Regina, Regina, Saskatchewan, Canada, caitlin.bakker@uregina.ca; 2Maastricht University, Maastricht, the Netherlands; 3Emory University, Atlanta, GA, US; 4University of Minnesota, Minneapolis, MN, US; 5BMJ, London, UK; 6London School of Hygiene and Tropical Medicine, London, UK; 7Amsterdam University Medical Center, Amsterdam, the Netherlands; 8Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.
Conflict of Interest Disclosures
Caitlin J. Bakker is cochair of the National Information Standards Organization Communication of Retractions, Removals and Expressions of Concern Standing Committee. Lex Bouter is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.
Funding/Support
This research is part of an ongoing PhD collaboration between The BMJ (British Medical Journal) and the team Meta-Research at Maastricht University (UM) on the responsible conduct of publishing scientific research. The BMJ is published by BMJ Group, a wholly owned subsidiary of the British Medical Association. UM is a public legal entity in the Netherlands. This study is part of Caitlin Bakker’s self-funded BMJ/UM PhD. No exchange of funds has taken place for this research project.
Role of the Funder/Sponsor
The authors are wholly responsible for the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.
Disclaimer
All authors express their own opinions and not necessarily that of their employers.