International Congress on
Peer Review and Scientific Publication

Enrique Vazquez,¹ Joseph S. Ross,^2,3,4 Cary P. Gross,^2,5,6 Karla Childers,⁷ Stephen Bamford,⁸ Joanne Waldstreicher,⁷ Harlan M. Krumholz,^3,4,9 Joshua D. Wallach¹⁰

Objective

The Yale Open Data Access (YODA) Project enables researchers to access shared participant-level clinical research data for independent secondary and replication studies.¹ The project, because it requires an application, provides an opportunity to determine how published analyses compare with the initial aims and the degree to which any deviance is noted in the publications. Accordingly, the objective of this study was to evaluate the concordance among the included trials, the study objectives, and the statistical methods specified in researchers’ requests to the YODA Project for Johnson & Johnson clinical trial data, the primary YODA data sharing partner, and their corresponding publications.

Design

In this cross-sectional study, all approved YODA requests for Johnson & Johnson pharmaceutical or medical device data that had 1 corresponding English-language publication or more were identified (from the first request in 2018 to October 29, 2021). From each request-publication pair, the primary objectives were classified as fully, partially, or not at all concordant. Primary and secondary end points were classified as fully concordant, partially concordant (≥ 1 additional primary or secondary end point in the request or publication), or discordant (≥ 1 secondary end point dropped or converted to a primary end point, primary end point converted to a secondary end point, or secondary and primary end points swapped). Given that slight methodological changes may have been necessary once researchers had access to the shared data, statistical methods were classified as concordant if the pairs described the same broad methodological approaches.

Results

Forty-eight requests on the YODA Project website with 1 publication or more in a peer-reviewed journal were identified. Of the 48 request-publication pairs, 33 (68.8%) had a fully concordant overarching study objective, and 13 (27.1%) had a partially concordant overarching study objective (Table 17). There were 28 pairs (58.3%) for which all of the requested trials were included in the analyses described in the publications; 17 pairs (35.4%) had articles that included fewer trials than the number of trials specified in the request. There were 31 pairs (64.6%) with fully concordant primary end points and 25 pairs (52.1%) with fully concordant secondary end points. Only 1 pair had fully concordant primary and secondary end points. Most pairs (39 [81.3%]) had concordant statistical methods; there were no pairs that were fully concordant across all proposal details.

Conclusions:

None of the YODA Project requests were fully concordant with their corresponding publications describing the completed research, most often because fewer trials were used than requested. These findings suggest that investigators using data from data sharing platforms should explain deviations from the data requests in their publications and that research reviewers should compare and evaluate the consistency between the prespecified requests and publications.

Reference

1. Ross JS, Waldstreicher J, Bamford S, et al. Overview and experience of the YODA Project with clinical trial data sharing after 5 years. Sci Data. 2018;5(1):180268. doi:10.1038/sdata.2018.268

¹Yale University, New Haven, CT, USA; ²Section of General Internal Medicine, Yale School of Medicine, New Haven, CT, USA;³Yale–New Haven Hospital Center for Outcomes Research and Evaluation, New Haven, CT, USA; ⁴Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA; ⁵Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University, New Haven, CT, USA; ⁶Department of Chronic Disease Epidemiology, Yale School of Public Health, New Haven, CT, USA; ⁷Johnson & Johnson, New Brunswick, NJ, USA; ⁸Johnson & Johnson, High Wycombe, UK; ⁹Section of Cardiovascular Medicine, Yale School of Medicine, New Haven, CT, USA; ¹⁰Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA, joshua.wallach@yale.edu

Conflict of Interest Disclosures

Joseph S. Ross and Harlan M. Krumholz report being cofounders of the Yale Open Data Access (YODA) Project, and Cary P. Gross and Joshua D. Wallach report being YODA project affiliates. Joseph S. Ross is a former associate editor of JAMA Internal Medicine and a current research editor at The BMJ and receives research support through Yale University from Johnson & Johnson to develop methods of clinical trial data sharing, from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology, from the US Food and Drug Administration (FDA) for the Yale–Mayo Clinic Center for Excellence in Regulatory Science and Innovation program (grant U01FD005938), from the Agency for Healthcare Research and Quality (grant R01HS022882), from the National Heart, Lung, and Blood Institute of the National Institutes of Health (NIH) (grants R01HS025164 and R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International; in addition, he is an expert witness at the request of the relator’s attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. Cary P. Gross has received research funding through Yale University from the National Comprehensive Cancer Network Foundation (funded by AstraZeneca) and Johnson & Johnson to help devise and implement new approaches to sharing clinical trial data and from Genentech. Karla Childers, Stephen Bamford, and Joanne Waldstreicher are employees and stockholders of Johnson & Johnson. Harlan M. Krumholz reports that he has contracts through Yale New Haven Hospital with the Centers for Medicare & Medicaid Services to support quality measurement programs and through Yale University with UnitedHealth Group to engage in collaborative research. He was a recipient of a research grant through Yale University from Medtronic for data sharing, from the FDA to develop methods for postmarket surveillance of medical devices, from Johnson & Johnson to support data sharing, and from the Shenzhen Center for Health Information for work to advance intelligent disease prevention and health promotion; he is an advisor to the National Center for Cardiovascular Diseases in Beijing, China; was an expert witness for the Arnold & Porter Law Firm for work related to the Sanofi clopidogrel litigation; and is an expert witness for the Martin/Baughman Law Firm for work related to the Cook Celect inferior vena cava (IVC) filter litigation and related to C. R. Bard Recovery IVC filter litigation and for the Siegfried and Jensen Law Firm for work related to Vioxx litigation; he chairs a cardiac scientific advisory board for UnitedHealth; was a member of the IBM Watson Health Life Sciences Board; is a member of the advisory board for Element Science, the health care advisory board for Facebook, and the physician advisory board for Aetna; he is the cofounder of HugoHealth, a personal health information platform, and cofounder of Refactor Health, an enterprise health care artificial intelligence–augmented data management company; he is a venture partner at F-Prime. Joshua D. Wallach is supported by the FDA and the National Institute on Alcohol Abuse and Alcoholism of the NIH under award 1K01AA028258.