Abstract
An Analysis of the History, Content, and Spin of Abstracts of COVID-19-Related Randomized Clinical Trials Posted as Preprints and Subsequently Published in Peer-Reviewed Journals or Unpublished
Hannah Spungen,1 Jason Burton,2 Stephen Schenkel,3 David L. Schriger1
Objective
Preprint servers have gained traction in many academic fields. Most preprints are unlikely to adversely affect public health. However, during a pandemic, when there is an urgent need for data, preprints may cause harm by disseminating incomplete, incorrect, or misleading information.1,2 The aim of this study was to characterize and compare the characteristics, completeness, and spin of the abstracts of all randomized clinical trials (RCTs) related to COVID-19 posted to medRxiv from March 13, 2020, to December 31, 2021. An additional aim was to identify all corresponding published versions of these abstracts and perform a similar qualitative comparison to examine the impact of the peer review process.
Design
An experienced librarian identified all COVID-19–related RCT preprints posted to medRxiv and all versions subsequently published in peer-reviewed journals as of June 1, 2022. An assistant created identically formatted Word documents of all abstracts. After training and the confirmation of adequate interrater reliability, 3 blinded reviewers scored individual abstracts presented in random order for completeness using items from CONSORT for Abstracts. They then evaluated blinded medRxiv/published abstract pairs for differences in content and spin, using criteria modified from Boutron et al.3 Last, the abstracts of all unpublished preprints, along with an equal-sized sample of subsequently published preprints and their published counterparts, were assessed for extent of spin. Analysis was descriptive.
Results
Two hundred ninety-one preprints were initially identified; 236 were confirmed as RCTs. Of these, 161 (68%) were found to have associated publications, which were published a median of 126 (IQR, 78-185; range, 0-654) days after medRxiv posting. The 75 unpublished preprints were posted a median of 344 (IQR, 235-429; range, 158-782) days prior to the final search. For most items, abstract completeness was higher in preprints that were subsequently published and was modestly higher still in published form (Table 9). The extent of spin was higher in unpublished preprints than in preprints that were subsequently published (Table 9). Last, of 161 published-preprint abstract pairs studied, 25% had more spin in the preprint version, 8% had more spin in the published version, and 66% had no difference in spin. Conversely, 12% had more consensible preprints, 42% had more consensible published versions, and 45% had no difference.
Conclusions
At this time, almost one-third of study medRxiv COVID-19–related RCTs have not been published, although roughly half were posted more than 1 year ago. This subset of unpublished preprints had lower CONSORT compliance and more spin than the medRxiv preprints that went on to be published. This study’s comparison of published-preprint pairs is consistent with other literature that shows that peer review and the publication process improves—but does not eliminate—incomplete reporting and spin. Limitations of this study include the restriction of the search to a single preprint server and the focus on abstracts alone.
References
1. Fraser N, Brierley L, Dey G, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLOS Biology. 2021;19(4):e3000959. doi:10.1371/journal.pbio.3000959.
2. Kodvanj I, Homolak J, Virag D, et al. Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues. Scientometrics. 2022;127(3):1339-1352. doi:10.1007/s11192-021-04249-7
3. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058-2064. doi:10.1001/jama.2010.651
1Department of Emergency Medicine, University of California, Los Angeles, CA, USA, hspungen@gmail.com; 2UCLA Library, University of California, Los Angeles, CA, USA; 3Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD, USA
Conflict of Interest Disclosures
None reported. David L. Schriger is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.
Funding/Support
David L. Schriger was supported in part by an unrestricted research grant from the Korein Foundation.