Abstract
Effect of Alerting Authors of Systematic Reviews and Guidelines That Research They Cited Had Been Retracted: A Randomized Controlled Trial
Alison Avenell,1 Mark J. Bolland,2 Greg D. Gamble,2 Andrew Grey2
Objective
Retracted clinical trials may be influential in systematic reviews and guidelines. Fanelli et al1 examined the impact of retractions on 50 meta-analyses and found little influence but suggested that the impact was likely variable and context specific. Using retracted clinical trial reports from one research group,2 responses of editors and authors to notifications that systematic reviews and guidelines they had published had cited the retracted trials were evaluated.
Design
Between November 2019 and January 2020, for 27 retracted trials (published 1997 to 2012, retracted 2016 to 2019) in osteoporosis and neurology, searches were conducted in Web of Science, Scopus, and researchers’ files for systematic reviews and clinical guidelines that cited the trials as evidence. Citing publications that acknowledged that they had cited retracted work were excluded. For each citing publication, 2 researchers independently coded the likely impact of removing the retracted trial reports, including findings likely to change (yes, no, or uncertain) and size of change (substantial, moderate, or minor). In a factorial design with 4 groups, authors of citing publications were randomized to receive up to 3 emails (if no reply) to the contact author and journal editor vs contact author only and, for citing publications with 2 or more authors, to an email to the contact author only vs up to 3 authors (selected from among contact, first, second, and final authors). Emails giving details of the retracted trials were sent monthly September to October 2020. Follow-up was 1 year for outcomes, assessed by replies to emails and notices in the public domain, including any reply (yes or no), time for a reply from first author, time for any reply, and action taken. Comparisons were undertaken using χ² tests. Email replies were analyzed for content by 3 researchers.
Results
A total of 88 citing publications (published 2003 to 2020) were identified; 2 were corrected before emails were sent. Authors/editors were emailed about 86 citing publications. A total of 45 of 88 citing publications (51%) had findings coded likely to change if the retracted trials were removed, and 39 of these (87%) were likely substantial impacts. Replies were received for 44 of 86 citing publications (51%). Emailing 3 authors was more likely to elicit a reply than emailing the contact author alone (26 of 42 [62%] vs 16 of 40 [40%]; P = .03), but including the editor did not increase replies (23 of 44 [52%] with editor vs 21 of 44 [48%] without editor; P = .66). Including more authors and/or the editor, whether findings were coded likely to change and the size of the likely change, had no effect on published corrective action. One year after emails were sent, only 9 publications had been corrected.
Conclusions
Retracted trials impact systematic reviews and guidelines. Emailing more authors to notify them of the retractions yielded more replies but did not increase corrections. Email alerts to authors and editors are inadequate to correct the impact of retracted publications in citing systematic reviews and guidelines. Publications with retracted citations should be marked until authors resolve concerns.
References
1. Fanelli D, Wong J, Moher D. What difference might retractions make? an estimate of the potential epistemic cost of retractions on meta-analyses. Account Res. Published online July 14, 2021. doi:10.1080/08989621.2021.1947810
2. Bolland MJ, Avenell A, Gamble GD, Grey A. Systematic review and statistical analysis of the integrity of 33 randomized controlled trials. Neurology. Published online November 9, 2016. doi:10.1212/WNL.0000000000003387
1Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, a.avenell@abdn.ac.uk; 2Department of Medicine, University of Auckland, Auckland, New Zealand
Conflict of Interest Disclosures
None reported.
Funding/Support
Mark J. Bolland is the recipient of an HRC Clinical Practitioner Fellowship. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates.