Abstract
Sustainable Approaches to Upholding High Integrity Standards in the Face of Large-Scale Threats: Insights From PLOS One
Renee V. Hoch,1 Emily J. Chenette1
Objective
PLOS One has experienced a sharp rise in paper mills and poor-quality submissions since 2021 and has taken an agile, multifaceted preventive approach to address these issues at scale. This quality improvement study evaluated trends in large-scale integrity issues (ie, megacases) at PLOS One between 2021 and 2024 and the impacts of preventive initiatives on the journal’s ability to address megacase concerns prior to publication.
Design
We analyzed temporal and geographic trends, issue types, and research areas for megacases detected at PLOS One between 2021 and 2024. We then evaluated changes to desk reject rates following specific interventions as well as changes to the pre- vs postpublication impacts of megacases over time.
Results
Between 2021 and 2024, PLOS One detected 50 megacases that cumulatively involved more than 7000 submissions and publications, with concerns including paper mills (42%; 21 of 50), authorship (22%; 11 of 50), peer review (52%; 26 of 50), and policy compliance (12%; 6 of 50). Megacases spanned multiple research areas, with the highest representation in the physical sciences and engineering (42%; 21 of 50) and medicine and public health (18%; 9 of 50). Most megacases involved submissions from geographically clustered author groups; megacases were received from 19 countries, with the highest representation from Pakistan (20%; 10 of 50), China (18%; 9 of 50), and India (10%; 5 of 50). The number of megacases detected by PLOS increased from 2 (214 articles) in 2021 to 25 (4310 articles) in 2024, and during this time the journal’s desk reject rate more than doubled, from 13% (5188 of 40,548) in 2021 to 27% (16,160 of 59,425) in 2024. In 2024 alone, we rejected more than 3000 submissions flagged for megacase concerns. Desk reject data in periods before and after specific preventive interventions indicated that each of these efforts increased the journal’s ability to detect and/or reject concerning content (Table 25-0967). We quantified pre- vs postpublication megacase impacts using a preventive efficacy metric, defined as the number of submissions flagged for megacase concerns (ie, prepublication impact) divided by the number of affected publications (ie, postpublication impact). PLOS One’s preventive efficacy increased from 0.70 in 2022 to 9.09 in 2024; the cumulative number of affected articles increased substantially during this time, but articles of concern were increasingly rejected prepublication, and the postpublication impact actually decreased (530 affected publications in 2022 vs 427 affected publications in 2024).
Conclusions
Large-scale integrity issues have risen considerably in recent years, affecting a broad array of subject areas and geographies. Through varied approaches, PLOS One has improved prepublication detection and rejection of content linked to megacases. This reduces the volume of unreliable publications, the resources required to address these integrity issues, and the impacts of megacases on peer review. Such scalable preventive approaches are crucial to the long-term sustainability of integrity work in an era of increasing paper mill activity.
Affiliations
1PLOS (Public Library of Science), Cambridge, UK, and San Francisco, CA, US, rhoch@plos.org.
Conflict of Interest Disclosures
Renee V. Hoch is an employee of PLOS; the head of the PLOS Publication Ethics team; a member of the Committee on Publication Ethics (COPE) Council, a United2Act Working Group, and an STM Task & Finish Working Group that developed guidance pertaining to large-scale integrity cases; a former member of the COPE Paper Mills Working Group; and a coauthor of COPE guidance on addressing large-scale paper mill issues. Emily J. Chenette is an employee of PLOS and is the editor in chief of PLOS One and head of PLOS editorial board services.
Acknowledgment
We thank the PLOS One Editorial and Publication Ethics teams for their work identifying and addressing large-scale cases, the PLOS editorial board management team for their help investigating and offboarding editorial board members of concern, and the PLOS Digital team for their help developing data analytics and technology tools needed for this work. Thanks also to Suzanne Farley, Rebecca Kirk, Zena Nyakoojo, and Dena Emmerson for their input on this abstract.
Additional Information
PLOS data and resources were used for this study, and others within PLOS Editorial provided input and contributed to the work underlying the datasets. PLOS was not otherwise involved in the reported study.