Abstract
An Update on Reporting Bias in the Antidepressant Literature: An FDA-Controlled Examination of Drug Efficacy
Erick H. Turner,1,2,3 Sepideh Alavi,2 Andrea Cipriani,4 Toshi Furukawa,5 Ilya Ivlev,1 Ryan McKenna,3 Yusuke Ogawa5
Objective
We previously investigated the influence of reporting bias on the apparent proportion of statistically significant trials and effect size estimates for antidepressant medications approved through 2004. We update those findings here for medications approved since 2004.
Design
We identified antidepressants approved by the US Food and Drug Administration (FDA) since 2004. We downloaded corresponding medical and statistical reviews from Drugs@FDA, identified phase 2 and 3 double-blind placebo-controlled efficacy trials, extracted summary statistics on each trial’s primary outcome, and extracted the FDA’s judgment as to whether each trial provided evidence of efficacy (statistical superiority to placebo on the primary outcome). For each FDA-registered trial, we searched the published literature for corresponding journal publications, extracted from the results sections summary statistics on the effect size for the stated primary outcome and whether the publication conveyed that the drug was effective, and compared trial outcome data from the FDA vs journal publications. We conducted 2 meta-analyses using the published literature and using FDA data and compared the resulting effect size (standardized mean difference (SMD) values using meta-regression. We repeated the meta-analysis comparison combining newer- and older-cohort datasets. We contrasted the extent of effect size inflation (bias) in the old vs new cohorts.
Results
Four antidepressant drugs were approved by the FDA since 2004: desvenlafaxine, levomilnacipran, vilazodone, and vortioxetine. As with older antidepressants, 50% of the FDA trials (n = 15) evaluating the newer drugs showed a statistically significant difference (Table). Reporting bias inflated the proportion of apparently positive trials (+21%), but less compared with the older cohort (+43%). Within the nonsignificant trials, the percentage published transparently (trial published and in agreement with FDA) increased significantly from 8.3% (older drugs) to 40% (newer drugs) (P = .01). Nevertheless, when nonsignificant and significant trials were combined, the rate of transparent publication was significantly greater for significant compared with nonsignificant trials (P = 5 ×10−19). In meta-analyses, the boost in SMD due to reporting bias diminished from 0.10 (older drugs) to 0.05 (newer drugs). Differences between FDA- and journal-based effect size values using meta-regression were statistically significant for the older drugs (P = .001), not statistically significant for newer drugs (P = .25), but statistically significant when older and newer drugs were combined (P = .003).
Conclusions
Reporting bias continues in the antidepressant clinical trial literature but findings with newer drugs compared with older drugs suggest a decrease in magnitude of reporting bias due to more transparent disclosure of nonstatistically significant clinical trial results.
1Oregon Health & Science University, Portland, OR, USA, eturnermd@gmail.com; 2VA Portland Health Care System, Portland, OR, USA; 3Scientific Resource Center, Agency for Health Research Quality, Portland, OR, USA; 4University of Oxford, Oxford, UK; 5Kyoto University Graduate School of Medicine, Kyoto, Japan
Conflict of Interest Disclosures:
Erick H. Turner receives protected time for research through Kyoto University Graduate School of Medicine. Andrea Cipriani has served as an expert witness for a patent litigation case about quetiapine extended release. Toshi Furukawa has received lecture fees from Eli Lilly, Janssen, Meiji, MSD, Otsuka, Pfizer and Tanabe-Mitsubishi and consultancy fees from Sekisui Chemicals; he has received royalties from Igaku-Shoin and Nihon Bunka Kagaku-sha publishers; and he has received research support from Mochida and Tanabe-Mitsubishi.