Abstract
Bias Associated With Publication of Interim Results of Randomized Trials: A Systematic Review
Steven Woloshin,1 Lisa M. Schwartz,1 Pamela J. Bagley,2 Heather B. Blunt,2 Brian White3
Objective
Publication of interim results from ongoing randomized clinical trials may generate substantial interest because they are new and often promising. Final results, however, may not confirm early promise and may receive less attention. Our objective was to describe the publication of interim results from randomized clinical trials and to compare the prominence and consistency with final publications.
Design
We conducted a PubMed search (2006-2015) for interim publications of randomized clinical trials, including the terms interim, not mature, or immature in the title or abstract. We used registration numbers and author names to search PubMed, ClinicalTrials.gov, and Web of Science through 2016 for final publications (authors were contacted if none identified) and determined each publication”s journal Impact Factor and Altmetric rating (ie, news and social media attention). Two researchers confirmed interim and final publication pairing and abstracted data.
Results
Of 1267 publications screened, 613 reported interim results (excluding completed pilot studies, protocols, and cancer trials reporting an interim result for the secondary outcome [overall survival] but with a final primary outcome result [progression-free survival]). Seventy-two percent (442 of 613) of these publications reported on trials stopped early (for benefit [105], harm [67], futility [224], other problems [46]). The remaining 171 ongoing trials reported interim efficacy or safety results. Forty percent (68 of 171) stated the reason for interim publication was a protocol-specified preplanned analysis; a few (6% [10 of 171 ]) stated other reasons (eg, response to release of results about the same intervention), but most (54% [93 of 171]) stated none. The 171 interim publications were mostly in oncology (28% [48]), surgery (18% [30]), or cardiology (11% [18]); 59% (101) had active controls, and 13% (23) tested noninferiority. The most commonly stated funding sources were solely industry (36% [61]), partly industry (10% [17]), government (18% [30]), and foundation or university (17% [29]). Final results were published for 57% (90) of the 158 trials where sufficient time elapsed for final publication (eg, >1 year beyond registry-specified study completion date). Most abstract conclusions (85% [61]) did not change qualitatively for the 72 pairs of interim and final publications reporting the same primary outcome results, while 15% changed: 8% (6) became weaker (eg, changed from “superior” to “not superior”), and 7% (5) became stronger. Interim and final publications had similar prominence in terms of Impact Factor and Altmetric rating (Table).
Conclusions
Frequent nonpublication of final results may cause bias because true treatment effects often remain unknown. Final publications, when available, have as much journal and media prominence as interim publications but may reach qualitatively different conclusions. Journals should publish fewer interim results (especially when not prespecified) and commit to making the final results known when they do.
1The Center for Medicine in the Media, Dartmouth Institute for Health Policy and Clinical Practice, Lebanon, NH, USA, lisa.schwartz@dartmouth.edu; 2Biomedical Libraries, Dartmouth College, Hanover, NH, USA; 3Atlanticare Regional Medical Center, Pomona, NJ, USA
Conflict of Interest Disclosures:
Drs Schwartz and Woloshin have served as medical experts in testosterone litigation. No other conflicts were reported.