International Congress on
Peer Review and Scientific Publication

Anisa Rowhani-Farid,¹ Kyungwan Hong,¹ Mikas Grewal,² Jesse Reynolds,³ Audrey D. Zhang,⁴ Joshua D. Wallach,⁵ Joseph S. Ross^6,7,8,9

Objective

Clinical trial integrity is compromised when investigators selectively report or misreport results, which leads to inaccurate claims of benefit and/or extrapolations from incomplete data.^1-3 This study examined the extent to which trials presented at major international medical conferences in 2016 consistently reported their study design, end points, and results across conference abstracts, published article abstracts, and press releases.

Design

A cross-sectional analysis of trials presented at 12 conferences in the US in 2016 was conducted. Conferences were identified from a list prepared by the Healthcare Convention and Exhibitors Association and were included if abstracts were publicly reported. From these conferences, all late-breaker trials were included and other trials were randomly selected, bringing the total sample to 25 abstracts per conference. First, it was determined whether trials were registered and reported results in an International Committee of Medical Journal Editors–approved trial registry. Second, it was determined whether trial results were published in a peer-reviewed journal. Finally, information on trial media coverage and press releases was collected using LexisNexis. For all published trials, the consistency of reporting of the following characteristics was examined, through comparison of the trials’ conference and publication abstracts: primary efficacy end point, safety end point, sample size, follow-up period, primary end point effect size, and characterization of results (comparisons were made for this characteristic across press releases too, if any). Authors determined consistency of reporting when identical information was presented across abstracts (and press releases). Primary analyses were descriptive; secondary analyses included χ² tests and multiple logistic regression.

Results

The sample comprised 240 trials presented at 12 conferences. Of these, 208 trials (86.7%) were registered, 95 (39.6%) reported summary results in a registry, and 177 (73.8%) were published; 82 trials (34.2%) were covered by the media, and 68 (28.3%) had press releases. Among the 177 published trials (Table 14 ), 171 (96.6%) reported consistent primary efficacy end points across abstracts, whereas 96 of 128 trials (75.0%) reported consistent safety outcomes. There were 107 of 172 trials (62.2%) with consistent sample sizes across abstracts, 101 of 137 trials (73.7%) that reported their follow-up periods consistently, 92 of 175 trials (52.6%) that described their effect sizes consistently, and 157 of 175 trials (89.7%) that characterized their results consistently. Among the trials that were published and had press releases, 32 of 32 (100%) characterized their results consistently across conference and publication abstracts and press releases. No trial characteristics were associated with reporting primary efficacy end points consistently.

Conclusions

This study demonstrates that trials are consistently reporting primary efficacy end points and results characterization. Lower consistency rates for other characteristics indicate that trial presentations are less likely to report safety end points and that authors could be presenting preliminary data at conferences with shorter follow-up periods and smaller sample sizes, owing to incomplete patient recruitment, and consequently, varying effect sizes. However, this does not rule out the possibility of misreporting at conferences or publications.

References

1. Boutron I. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA.2010;303:2058-2064. doi:10.1001/jama.2010.651

2. Chiu K, Grundy Q, Bero L. “Spin” in published biomedical literature: a methodological systematic review. PLoS Biol.2017;15:e2002173. doi:10.1371/journal.pbio.2002173

3. Diong J, Butler AA, Gandevia SC, Héroux ME. Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PLoS One. 2018;13:e0202121. doi: 10.1371/journal.pone.0202121

¹Restoring Invisible and Abandoned Trials Support Center, Department of Pharmaceutical Health Services Research, University of Maryland, Baltimore, MD, USA, arowhani@rx.umaryland.edu; ²Yale School of Medicine, New Haven, CT, USA; ³Yale School of Public Health, New Haven, CT, USA; ⁴Department of Internal Medicine, Duke University School of Medicine, Durham, NC, USA; ⁵Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA; ⁶Section of General Internal Medicine, Yale School of Medicine, New Haven, CT, USA; ⁷National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, USA; ⁸Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA; ⁹Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA

Conflict of Interest Disclosures

The Laura and John Arnold Foundation funds the Restoring Invisible & Abandoned Trials Support Center, which supports the salaries of Anisa Rowhani-Farid and Kyungwan Hong. Kyungwan Hong was supported by the Food and Drug Administration (FDA) of the US Department of Health and Human Services (HHS) as part of a financial assistance award U01FD005946, unrelated to this manuscript, totaling $5000 with 100% funded by the FDA/HHS. Audrey D. Zhang currently receives research support from the National Institute on Aging through the Duke Creating Alzheimer’s Disease and Related Dementias Researchers for the Next Generation—Stimulating Access to Research in Residency (CARiNG-StARR) program (R38AG065762). Joshua D. Wallach is supported by the FDA and the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health (NIH) under award 1K01AA028258. Joseph S. Ross currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology, from the FDA for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation program (U01FD005938), from the Agency for Healthcare Research and Quality (R01HS022882), from the National Heart, Lung, and Blood Institute of the NIH (R01HS025164, R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International; in addition, Joseph S. Ross is an expert witness at the request of relator’s attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. No other disclosures were reported.

Additional information

Kyungwan Hong is a co–corresponding author.