Changes to Research Article Abstracts Between Submission and Publication

Christos P. Kotanidis,^1,2,3 Sarah Gorey,^1,4 Harleen Marwah,¹ Abarna Pearl,¹ Darren Taichman,¹ Mary Beth Hamel^1,5

Objective

Editorial review forms the cornerstone of scientific publishing, ensuring that published research adheres to the highest quality standards.¹ We evaluated how editorial processes shape manuscripts by examining changes in abstracts between submission and publication.

Design

We assessed research articles submitted to the New England Journal of Medicine (NEJM) in 2022 and published in NEJM or rejected and subsequently published in 437 other journals. Using author names, we searched PubMed for articles within 2 years of submission. Four physicians reviewed search results to match published manuscripts with submitted manuscripts. Abstract similarity was quantified using term frequency–inverse document frequency,^2,3 assigning a cosine similarity score ranging from 0 (dissimilar) to 1 (identical). To assess if similarity changes reflected substantive differences, masked physician reviewers conducted qualitative comparisons of submitted and published versions of abstracts from clinical trials. Abstracts were assessed for substantive changes in the following 4 domains: trial design, primary outcome, adverse events, and conclusion (TPAC). A score of +1 was assigned for a substantive domain change favoring the published version and −1 for domain change favoring the submitted version, with the domain scores summed, resulting in a total TPAC score range of −4 (submitted better) to +4 (published better).

Results

We matched 1360 PubMed records from 2756 research articles initially submitted to NEJM. Of these, 201 were published in NEJM and 1159 were rejected by NEJM and published in the 437 other journals. The median (IQR) similarity score between submitted and published abstracts was 0.71 (0.58-0.83). Abstracts of articles published in the 5 general medicine journals with a journal impact factor (JIF) above 50 per the 2023 Clarivate report had significantly lower similarity scores compared with those published in other journals (Figure 25-0940, A). For qualitative comparisons, we identified abstracts from randomized controlled trials. On average, published abstracts improved in 0.9 domains, with conclusion being the most frequently modified domain (44.2%). Similarity score correlated negatively with TPAC (Spearman ρ coefficient, −0.39; Figure 25-0940, B), indicating that lower similarity reflected more substantive revisions. Among abstracts published in the 5 high-JIF general medicine journals, 72.1% improved in at least 1 domain, compared with 48.3% of abstracts published in the other 433 journals. This difference was consistent across all TPAC domains, with conclusion changes being the most different (48% vs 27%).

Conclusions

Our comparisons of submitted and published abstracts suggest that peer review and editorial processes result in substantial revision of research reports. This report focused on submissions to 1 general medical journal and within a 2-year publication window. These findings reinforce the importance of editorial oversight in scientific publishing, highlighting its role in refining study design communication, reporting of outcomes, and overall clarity in research dissemination.

References

1. Roll SC. The value and process of high-quality peer review in scientific professional journals. J Diagn Med Sonography. 2019;35(5):359-362. doi:10.1177/8756479319853800

2. Aizawa A. An information-theoretic perspective of tf–idf measures. Inf Process & Manage. 2003;39(1):45-65. doi:10.1016/S0306-4573(02)00021-3

3. Ramos J. Using tf-idf to determine word relevance in document queries. In: Proceedings of the First Instructional Conference on Machine Learning. 2003.

¹New England Journal of Medicine, Boston, MA, US, ckotanidis@ nejm.org; ²Heart and Vascular Center, Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; ³Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, Oxford, UK; ⁴Stroke Clinical Trials Network Ireland (SCTNI), University College Dublin Clinical Research Centre, Mater Hospital, Dublin, Ireland; ⁵Division of General Medicine and Primary Care, Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, US.

Conflict of Interest Disclosures

Christos P. Kotanidis, Sarah Gorey, Harleen Marwah, and Abarna Pearl are Editorial Fellows, Darren Taichman is a Deputy Editor, and Mary Beth Hamel is the Executive Editor at the New England Journal of Medicine.