Abstract
A Comprehensive Assessment of Changes to Prespecified Trial Outcomes, Including Historical Registry Records
Martin R. Holst,1 Martin Haslberger,1 Daniel Strech,1 Lars G. Hemkens,2,3,4 Benjamin G. Carlisle1
Objective
Preregistration plays a key role in reducing certain biases in clinical trial conduct and reporting. Most studies on selective reporting have assessed discrepancies between the latest registry entry and the corresponding publication. However, ClinicalTrials.gov and the German Clinical Trials Register (DRKS) allow entries to be updated after initial registration, which allows researchers to add or change crucial details. Because studies look at only the latest registry entry, these later changes are not directly visible, which constitutes another source of possible reporting bias if larger changes go unreported. By assessing the entire audit trail, which was not easily accessible for mass download and analysis until recently, this study was a comprehensive analysis of outcome changes made within registry entries between key study time points and in corresponding results publications and analyzed factors associated with these changes.
Design
Based on an existing data set of all trials conducted at German university medical centers between 2009 and 2017 and registered in DRKS or ClinicalTrials.gov that had published results,1,2 all historical registry entries were obtained using the R package cthist.3 Historical trial registration records from the 2 databases were extracted and semiautomatically evaluated. For each trial, changes to primary outcomes between key study time points in the registry (first patient inclusion, completion, publication, and latest available version) and publication were extracted. These changes were then classified according to their severity, and associations with a number of candidate predictors of outcome changes at different trial stages were analyzed.
Results
Of 1747 included trials, 592 trials (33.9%) had an outcome change of some kind within the registration after study start. Analyses of outcomes in publications and the nature and severity of outcome changes are ongoing, and results will be presented at the conference. The study also examined whether and how outcome changes were reported in results publications and presented factors associated with outcome changes at different trial stages.
Conclusions
A large proportion of clinical trials exhibited changes to prespecified outcomes within the registry after study start, and the nature of these changes is important to know for the integrity of the scientific process. This analysis provided further insight into outcome registration and reporting practices. Using methods reported in this study, peer reviewers, editors, readers, and metaresearchers may be able to assess the registration quality of a clinical trial.
References
1. Riedel N, Wieschowski S, Bruckner T, et al. Results dissemination from completed clinical trials conducted at German university medical centers remained delayed and incomplete: the 2014–2017 cohort. J Clin Epidemiol. 2022;144:1-7. doi:10.1016/j.jclinepi.2021.12.012
2. Wieschowski S, Riedel N, Wollmann K, et al. Result dissemination from clinical trials conducted at German university medical centers was delayed and incomplete. J Clin Epidemiol. 2019;115:37-45. doi:10.1016/j.jclinepi.2019.06.002
3. Carlisle BG. Analysis of clinical trial registry entry histories using the novel R package cthist. medRxiv. Preprint posted online January 21, 2022. doi:10.1101/2022.01.20.22269538
1Berlin Institute of Health at Charité–Universitätsmedizin Berlin, QUEST Center for Responsible Research, Berlin, Germany, martin.haslberger@bih-charite.de; 2Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; 3Berlin Institute of Health at Charité–Universitätsmedizin Berlin, QUEST Center for Responsible Research, Meta-Research Innovation Center Berlin, Berlin, Germany; 4Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA
Conflict of Interest Disclosures
The authors declare no direct conflicts of interest. Daniel Strech is a member of the Sanofi Advisory Bioethics Committee and receives an honorarium for his contribution to meetings.
Funding/Support
The project is funded from QUEST departmental resources.
Additional Information
This study protocol was preregistered (https://osf.io/t3qva). Martin R. Holst and Benjamin G. Carlisle are co–corresponding authors.