Abstract
Outcome Switching in Observational Studies of Interventions: Comparison of Registration Records and Published Articles
Zexing Song,1,2 Cecilie Jespersen,3,4 Asbjørn Hróbjartsson,3,4 S. Joseph Kim,1,5,6 Rob Fowler,1,5 Peter C. Austin,1,6 An-Wen Chan1,2,5
Objective
Outcome switching between study design and reporting is a potential source of bias in observational studies, but there is a paucity of evidence as to its frequency. We aimed to estimate the prevalence of outcome switching in observational studies of interventions (defined as controlled cohort studies investigating the causal effects of interventions on health-related outcomes). Secondary aims included assessing the completeness of prespecification of primary outcomes and factors associated with outcome switching.
Design
This meta-epidemiological study involved longitudinal analyses of observational studies of interventions prospectively registered on ClinicalTrials.gov within 1 month of their study start date between 2014 and 2016 that had results published in a peer-reviewed journal. We screened registry records from January through December 2024 to create the study sample and completed outcome data extraction and analysis from January through April 2025. Complete outcome prespecification required explicit definition in the registry of the measurement variable, analysis metric, method of aggregation (the statistic to summarize the outcome within each group), and time point of the outcome. We evaluated outcome switching by identifying discrepancies in the primary outcomes between the registry and published articles, including omission (prespecified primary outcomes not reported), downgrading (prespecified primary outcomes reported as nonprimary), upgrading (prespecified nonprimary outcomes reported as primary), and introduction of new primary outcomes not listed in the registry. We considered outcome switching to favor statistically significant results if a new statistically significant primary outcome was introduced or upgraded or a nonsignificant one was downgraded. We performed multivariable logistic regression to estimate the association between study characteristics and outcome switching.
Results
We screened 9965 registry records labelled as observational studies and included 127 eligible studies with results published between January 2015 and October 2024. Only 23 studies (18%) completely prespecified their primary outcome in the registry, and the method of aggregation was the least commonly defined element (33 [26%]). Outcome switching was found in 60 studies (47%), and only 1 of these studies (2%) provided a rationale for the changes. The most common discrepancy was omission (32 [25%]) followed by downgrading (30 [24%]), introduction of new primary outcomes (23 studies with 29 new primary outcomes [18%]), and upgrading (2 [2%]). New primary outcomes differed most commonly between the registry and published articles in the measurement variable (21 of 29 [72%]) and time point (15 of 29 [52%]). Among 54 studies that had discrepancies not limited to omitted primary outcomes, statistically significant results were favored in 80% (43 of 54). No study characteristics were significantly associated with outcome switching (Table 25-0874).
Conclusions
Unreported outcome switching and inadequate outcome prespecification were common in observational studies of interventions. These findings underscore the need for improved registration practices and greater transparency to better understand the risk of bias in observational research.
1Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada, zexing.song@mail.utoronto.ca; 2Division of Dermatology, Women’s College Research Institute, Women’s College Hospital, Toronto, Ontario, Canada; 3Cochrane Denmark & Centre for Evidence-Based Medicine Odense (CEBMO), University of Southern Denmark, Odense, Denmark; 4Open Patient data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark; 5Department of Medicine, University of Toronto, Toronto, Ontario, Canada; 6ICES, Toronto, Ontario, Canada.
Conflict of Interest Disclosures
An-Wen Chan is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.