Abstract
In-Person Peer Review Training to Improve Preparedness to Evaluate Manuscripts
Marcus Tolentino Silva,1 Taís Freire Galvão2,3
Objective
Peer review is the core of scientific publication, and mentoring seems to enhance the quality of reviews.1 We aimed to assess the effect of peer review training on reviewers’ preparedness to evaluate manuscripts.
Design
This study was designed to improve peer review quality for a diamond open access journal, Epidemiologia e Serviços de Saúde: revista do SUS (RESS). The intervention consisted of a 16-hour, in-person, 2-day workshop.2 A checklist consisting of 31 critical, important, or desirable items for the preparation of peer review opinions was developed, and its use was encouraged to guide reports.3 The courses took place in June (including associated editors and reviewers), September, and October 2024 in 3 Brazilian cities and consisted of practical sections of peer review, preceded by an explanation of the checklist and peer review process. Instructors were available for students’ questions, and discussions about bias in scientific communication were stimulated. Before starting, participants were invited to answer an anonymized questionnaire to assess their reviewing skills by means of a 5-point Likert scale (1: poor, 5: excellent), and another survey was sent electronically in January 2025 to assess checklist usefulness and the mean difference (MD) and 95% confidence interval (95% CI) of pre- and post-event confidence as a peer reviewer. We described participants’ characteristics and summarized means and standard deviations (SDs), and tested differences in distribution using Fisher exact, analysis of variance, Kruskal-Wallis, or t tests. All analyses were performed in Stata v.14.2. The project was approved as an extension activity of the University of Brasília, where all participants registered themselves.
Results
In total, 96 participants were trained; 51 (76.1%) were women and 38 (56.7%) had a doctoral degree, which was more frequent in first-edition participants (26 [70.3%]; P = .02). For 67 initial survey respondents, the mean (SD) familiarity with article submission was 4.1 (1.1) and with peer review, 3.5 (1.4). Mean (SD) confidence as a peer reviewer was 3.5 (1.3). The January 2025 survey had 43 respondents, who reported having issued a mean (SD) of 7.6 (25.9) peer reviews since the workshops. Confidence as a peer reviewer significantly increased (MD, 0.5 [95% CI, 0.1 to 1.0]; P = .01); 86.1% (n = 37) reported using the checklist to issue peer reviews and gave it a mean (SD) rating of 4.6 (0.9). Results are summarized in Table 25-0888.
Conclusions
Peer review practical training sections and a checklist with suggested items improved peer reviewers’ confidence in evaluating manuscripts. Taking into account the costs and sustainability issues involved, this strategy may be viable to improve peer reviews.
References
1. Lyons-Warren AM, Aamodt WW, Pieper KM, Strowd RE.
A structured, journal-led peer-review mentoring program enhances peer review training. Res Integr Peer Rev. 2024;9(1):3. doi:10.1186/s41073-024-00143-x
2. Silva MT, Galvão TF. Systematization of peer review in Epidemiologia e Serviços de Saúde. Epidemiol Serv Saude. 2024;33:e20241001. doi:10.1590/S2237-96222024v33e20241001.en
3. Silva MT, Galvão TF. Revisão por pares: itens recomendados na elaboração de pareceres [Peer review: recommended items for preparing opinions]. June 19, 2024. Accessed May 24, 2025. https://osf.io/grn2a
1Faculdade de Ciências da Saúde, Universidade de Brasília, Brasília, Brazil; 2Faculdade de Ciências Farmacêuticas, Universidade Estadual de Campinas, Campinas, Brazil, taisgalvao@gmail.com; 3Secretaria de Vigilância em Saúde e Ambiente, Ministério da Saúde, Brasília, Brazil.
Conflict of Interest Disclosures
Taís Freire Galvão reports being an executive editor of Epidemiologia e Serviços de Saúde: revista do SUS (RESS). Taís Freire Galvão is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.