Abstract
Differences Between Manuscript Versions:
A Living Review and Series of Meta-Analyses
Mario Malički,1,2,3 Ana Jerončić,4 Gerben ter Riet,5,6 Lex Bouter,7,8 John P. A. Ioannidis,2,3,9,10 IJsbrand Jan Aalbersberg,11 Steven N. Goodman1,2,3,9,12
Objective
Previous research has indicated a knowledge gap on changes brought on by peer review and journal publishing.1 We are conducting a living evidence synthesis of studies that analyzed differences between manuscript versions (eg, preprinted or submitted versions vs peer reviewed journal versions).
Design
This is a living synthesis of studies with meta-analyses of proportions (following PRISMA guidelines for reporting).2 Studies were identified based on authors’ knowledge of the field, from 18 previous systematic reviews on peer review,3 and by checking all research presented at the peer review conferences. All references and citations of identified studies (using Google Scholar up to January 2025) were checked. For all included studies that reported on changes, we extracted descriptive variables (including year of publication, sampling method, change comparison method, and data, wherever available, on how many analyzed version-pairs had changes for 11 outcomes). We assessed risk of bias for each outcome based on 3 domains: sample selection, measurement reliability, and conflicts of interest. To pool the data, we used random effects meta-analysis with Freeman-Tukey transformed proportions and reported the I2 index of heterogeneity.
Results
We identified 67 studies published from 1978 through the end of 2024, of which 31 (46%) analyzed changes between preprint and journal versions, 23 (34%) between submitted and accepted or published versions, 8 (12%) between rejected versions and those later published in other journals, 4 (6%) between multiple version sets, and 1 (1.5%) between different preprinted versions. Most studies, 45 (67%), analyzed changes manually, 13 (19%) used computational methods, and 9 (12%) combined manual and computational methods. The median number of analyzed version-pairs was 113 (IQR, 51-429). By discipline, 43 (64%) studies looked only at health research, 6 (9%) at life sciences, 6 (9%) at social sciences, 4 (6%) at physical sciences, and 8 (12%) at multiple disciplines. Meta-analyses showed the highest frequency of changes in title, authorship, conflicts of interest, and numerical results, with the lowest frequency in study conclusions (Table 25-1077).
Conclusions
Current evidence indicates that while different manuscript sections experience varying degrees of changes from their submitted, preprinted, or rejected versions to their peer-reviewed journal version, the primary conclusions rarely change. Our results in this stage are limited by not conducting a systematic search of bibliographic databases and lack of (identified) data points from many subdisciplines, especially arts and humanities, as well as studies that explored possible factors associated with the extent of changes (eg, researchers’ or editors’ career stage or prestige, quality of initial manuscript version).
References
1. Tennant JP, Ross-Hellauer T. The limitations to our understanding of peer review. Res Integr Peer Rev. 2020;5:6. doi:10.1186/s41073-020-00092-1
2. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71
3. European Association of Science Editors. What is peer review? December 28, 2024. Accessed July 15, 2025. https://ease.org.uk/communities/peer-review-committee/peer-review-toolkit/What-is-peer-review/
1Stanford Program on Research Rigor and Reproducibility (SPORR), Stanford University, Stanford, California, US, mmalicki@stanford.edu; 2Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, US; 3Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US; 4Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia; 5Urban Vitality Centre of Expertise, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands; 6Amsterdam University Medical Centers, Department of Cardiology, Amsterdam, The Netherlands; 7Department of Philosophy, Faculty of Humanities, Vrije Universiteit, Amsterdam, The Netherlands; 8Amsterdam University Medical Centers, Department of Epidemiology and Data Science, Amsterdam, The Netherlands; 9Department of Medicine, Stanford University School of Medicine, Stanford, California, US; 10Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, US; 11Independent scholar; 12Department of Health Policy, Stanford University School of Medicine, Stanford, California, US.
Conflict of Interest Disclosures Mario Malički was a co–editor in chief of Research Integrity and Peer Review, in which 1 of the included studies was published (he was not involved in the processing or decision making for that study, and Research Integrity and Peer Review full peer review histories are available to the public on the journal website). Mario Malički was also an external reviewer for 1 study. Gerben ter Riet is a co-author of 1 of the included studies and Steven N. Goodman is a co-author of 1 of the included studies. In both of these cases, data extractions and primary data analyses were done by Mario Malički and Ana Jerončić. IJsbrand Jan Aalbersberg was a Senior Vice President of Research Integrity at Elsevier. John P. A. Ioannidis, Lex Bouter, and Steven N. Goodman are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract.
Funding/Support
The work on this project started in 2020, and in 2020 Elsevier funding was awarded to Stanford University for a 1-year METRICS postdoctoral position that supported Dr Malički’s work. From 2021 to 2025, Dr Malički’s work has been supported by the Stanford School of Medicine Research Office, which also supports the work of Dr Goodman. At the start of the project, IJsbrand Jan Aalbersberg was an employee of Elsevier.
Acknowledgments
In 2022’s Peer Review Congress, we presented preliminary findings (https://peerreviewcongress.org/abstract/a-synthesis-of-studies-on-changes-manuscripts-underwent-between-submission-or-preprint-posting-and-peer-reviewed-journal-publication/) that covered 25 studies published between 1990 and 2021. The current results include studies published before 1990 and up to January 2025.
Additional Information
A website with computationally reproducible results from this living review will be available soon.