Effects of Peer Review and Editorial Workflows in Decision-Making at a Diamond Open Access Journal
Abstract
Tais Freire Galvão,1,2 Everton Nunes da Silva,2 Jorge Otávio Maia Barreto,3 Marcus Tolentino Silva4
Objective
Detailed descriptions of journals’ workflows exist,1 but empirical studies assessing their effects are scarce. In 2024, a new workflow was implemented at Epidemiologia e Serviços de Saúde: revista do SUS (RESS) aimed at getting timely and quality reviews. This study assessed the effect of a new workflow on the journal’s time to decision-making.
Design
This before and after study using historical controls assessed submissions to RESS from June to December 2024 (intervention) and June to December 2023 (control), followed up until January 2025. The intervention started in June 2024, based on inclusion of a peer review administrator in charge of reviewers’ assignments and adding new and trained associate editors2; previous evaluation by the editor in chief (scope) and scientific editor (methodological quality); and checklists to standardize reviews.3 The comparator was the previous workflow, centralized on the associate editor’s assignment of reviewers, and final decisions made by the editor in chief and a member of the editorial committee. Outcomes were processing time from submission to decision, number of reviewers invited, and reviews received. We summarized means and SDs or medians and IQRs. Normality was tested using the Shapiro-Wilk test and square root or logarithmic transformations preceded t-tests, with the Levene test used to assess variances’ homogeneity. In unsuitable t-tests, the Mann-Whitney test was applied. Residual normality and heteroscedasticity were examined by Breusch-Pagan test. We adjusted by the length of the paper (full or short) to minimize potential confounding.
Results
Out of 928 submissions in 2024, 218 proceeded to peer review (140 ongoing until cohort closure) and 78 had a decision (36 accepted, 42 rejected). In 2023, from 970 submissions, 77 were peer reviewed and resulted in 28 acceptances and 49 rejections (P = .22). Mean (SD) time in days to decision was significantly shorter in the intervention (62.8 [40.5]) than in the controls (130.1 [64.5]; P < .001), which was also faster in the intervention according to acceptance (92.8 [39.8] vs 133.5 [61.6]; P = .002) or rejection (37.1 [16.0] vs 128.1 [66.7]; P < .001). The mean (SD) number of invited reviewers per manuscript was higher in the intervention (10.1 [6.3]) than in the controls (7.8 [5.5]; P = .02), without significant difference according to decision when compared with the historical controls (approved, 10.0 [7.0] vs 6.3 [2.8]; P = .07; rejected, 10.3 [5.8] vs 8.6 [6.5]; adjusted P = .11). The median (IQR) number of reviews received was also higher (4 [4-4] vs 2 [2-3]; P < .001), and was similarly higher when comparing decisions across groups (approved, 4 [4-4] vs 2 [2-3]; P < .001; rejected, 4 [4-4] vs 2 [2-3]; P < .001) (Table 25-0909).
Conclusions
The new workflow reduced the duration of editorial processing and allowed a higher number of peer reviews to inform decision-making. The outcomes were not defined before the intervention was planned; the length of follow-up, new editors, and submission system were confounders that may have influenced the findings.
References
1. Mark H, Ragon T, Funning G, et al. Editorial workflow of a community-led, all-volunteer scientific journal: lessons from the launch of Seismica. Seismica. 2023;2(2):1091. doi:10.26443/seismica.v2i2.1091
2. Galvão TF, Silva EN, Araújo WN, Barreto JOM. Editorial improvement in Epidemiologia e Serviços de Saúde in 2024. Epidemiol Serv Saude. 2024;33:e20241002. doi:10.1590/S2237-96222024v33e20241002.en
3. Silva MT, Galvão TF. Systematization of peer review in Epidemiologia e Serviços de Saúde. Epidemiol Serv Saude. 2024;33:e20241001. doi:10.1590/S2237-96222024v33e20241001.en
1Faculdade de Ciências Farmacêuticas, Universidade Estadual de Campinas, Campinas, Brazil, taisgalvao@gmail.com; 2Secretaria de Vigilância em Saúde e Ambiente, Ministério da Saúde, Brasília, Brazil; 3Faculdade de Ciências e Tecnologias em Saúde, Universidade de Brasília, Brasília, Brazil; 4Gerência Regional de Brasília, Fundação Oswaldo Cruz, Brasília, Brazil; 5Faculdade de Ciências da Saúde, Universidade de Brasília, Brasília, Brazil.
Conflict of Interest Disclosures
Tais Freire Galvão, Everton Nunes da Silva, and Jorge Otávio Maia Barreto are editors of Epidemiologia e Serviços de Saúde: revista do SUS. Marcus Tolentino Silva has no conflicts of interest to declare. Tais Freire Galvão is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.