Abstract
Inclusion of Randomized Controlled Trials Published by Potentially Predatory Journals in Anesthesiology Systematic Reviews: A Cross-Sectional Study
Julián C. Velásquez Paz,1 Andrés Zorrilla Vaca,2 Markus Klimek,3 Jose A. Calvache1,3
Objective
In recent years, systematic reviews (SRs) and meta-analyses (MAs)—traditionally regarded as reliable sources of causal evidence due to their methodological rigor—have increasingly been compromised by the inclusion of predatory literature, which is marked by misleading information and questionable editorial practices.1 Despite ongoing efforts,2 predatory literature continues to pose a major challenge in evidence synthesis. We aimed to assess the degree of inclusion of randomized controlled trials (RCTs) published in potentially predatory journals into SRs with MAs published in leading anesthesiology journals.
Methods
We conducted a cross-sectional study in which we included SRs with MAs published between 2021 and 2022 in the top 10 high-ranked anesthesia journals based on the Web of Science’s Journal Citation Report for 2021. Exclusion criteria included SRs without MAs or network MAs as well as those including observational studies and nonhuman studies. To determine whether an RCT was published in a potentially predatory journal, 2 authors independently followed a stepwise process: (1) an initial assessment of the publisher’s reputation, confirming if it was widely recognized and reputable in the academic publishing community; (2) verification of the journal’s indexing in the Committee on Publication Ethics or Directory of Open Access Journals; and (3) application of the Loyola Marymount University Journal Evaluation Tool (JET).3 Discrepancies between the 2 authors were resolved by consensus. The JET evaluates journals based on key criteria such as transparency of editorial processes, rigor of peer review practices, adherence to ethical standards, presence in reputable indexing databases, archiving practices, and the quality of the journal’s website; each aspect is scored using defined indicators based on their credibility and risk of predatory practices (Figure 25-1165). We summarized our results using descriptive statistics.
Results
A total of 79 SRs with MAs including 1110 RCTs were analyzed, with a median of 10 RCTs per SR. Most studies originated from the US, China, and India (432 [38.9%]). After applying the stepwise approach, 29 (2.6%) of the analyzed RCTs were identified as published in journals classified by our process as untrustworthy; in addition, RCTs published in potentially predatory journals were included in 14 (17.7%) of the 79 studied SRs. Finally, after reviewing Beall’s list, www.predatoryjournals.org, and journals excluded from the Web of Science or Scopus since 2017, we found that only 5 (17.2%) of the RCTs published in journals classified as untrustworthy appeared on any of these lists.
Conclusions
This study highlights that RCTs published in potentially predatory journals are being cited in SRs with MAs published in leading anesthesiology journals. Our findings indicate that while lists of potentially predatory journals can be helpful, they are probably not sufficient to fully identify actual predatory journals. Further analysis is needed to determine the impact of RCTs published by potentially predatory journals on the main findings of these MAs.
References
1. Grudniewicz A, Moher D, Cobey KD, et al. Predatory journals: no definition, no defence. Nature. 2019;576(7786):210-212. doi:10.1038/d41586-019-03759-y
2. Cukier S, Helal L, Rice DB, et al. Checklists to detect potential predatory biomedical journals: a systematic review. BMC Med. 2020;18(1):104. doi:10.1186/s12916-020-01566-1
3. Rele S, Kennedy M, Blas N. Journal Evaluation Tool. Loyola Marymount University. 2017. Accessed July 10, 2024. https://digitalcommons.lmu.edu/librarian_pubs/40
1Department of Anaesthesiology, Universidad del Cauca, Popayan, Colombia, jacalvache@unicauca.edu.co; 2Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Boston, MA, US; 3Department of Anesthesiology, Erasmus University Medical Center Rotterdam, the Netherlands.
Conflict of Interest Disclosures
None reported.