Author Practices and Experiences With PRISMA-P 2015

Mette B. Engmose,^1,2 An-Wen Chan,³ Kerry Dwan,⁴ Carsten Hinrichsen,⁵ Asbjørn Hróbjartsson,^1,2 David Moher,⁶ Matthew J. Page,⁷ Larissa Shamseer,⁸ Lesley A. Stewart,⁴ Camilla H. Nejstgaard^1,2

Objective

Systematic review protocols often adhere incompletely to the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015.¹ Reporting guidelines need to be regularly updated to incorporate methodological developments, author feedback, and changed research context. We aimed to inform the update of PRISMA-P 2015² by exploring protocol authors’ practices and experiences with the reporting guideline.

Design

In a cross-sectional study, we investigated adherence to PRISMA-P 2015 and interviewed protocol authors. We randomly sampled 100 systematic review protocols from May 2021 to May 2024: 50 protocols published in PubMed-indexed journals and 50 protocols uploaded to or registered in OSF/PROSPERO. Two authors independently assessed the 26 PRISMA-P 2015 items as fully, partially, or not reported (or not applicable). We analyzed protocol adherence overall and at the item level. We invited 43 corresponding authors from our sample of 100 systematic review protocols for interviews on their experiences with using PRISMA-P 2015. The selection of authors was based on maximum variation and snowball sampling and personal network. We used a piloted, semistructured interview guide involving 3 predefined themes: level of experience, views on using PRISMA-P 2015, and reflections on the guideline’s strengths and weaknesses. We applied framework analysis to the interview transcripts. The study was reported according to Standards for Reporting Qualitative Research (SRQR).³

Results

The PubMed-indexed protocols fully adhered to a median of 60% (range, 36%-80%) of PRISMA-P 2015 items. The equivalent median for OSF/PROSPERO protocols was 45% (range, 25%-88%). In both types of protocols, items not adhered to in more than 25% of the protocols were related to protocol amendments (item 4), role of the funder (item 5c), methods for planned summary other than quantitative (item 15d), meta-bias(es) (item 16), and confidence in the cumulative evidence (item 17) (Table 25-1033). In both types of protocols, 11 items partially adhered to in more than 25% of the protocols were lacking, eg, descriptions of procedures for data selection from multiple reports of the same study (item 11c). Of the 43 invited authors, 15 (9 men, 6 women) participated in the interviews. From the predefined themes, several suggestions for the PRISMA-P 2015 update emerged. Some suggestions regarded adding or modifying existing content, eg, to report conflicts of interest or how to report data synthesis when no meta-analysis was planned; other suggestions were more generic, eg, to add links to the Elaboration & Explanation paper.

Conclusions

Adherence to PRISMA-P 2015 was inadequate in 50 systematic review protocols from PubMed-indexed journals and especially in 50 protocols from OSF/PROSPERO. Aspects often not adhered to were protocol amendments, role of the funder, methods for planned summary other than quantitative, meta-bias(es), and confidence in the cumulative evidence. The interviewed authors suggested several modifications of the guideline. Findings from this study will inform the update of PRISMA-P 2015.

References

1. Frost AD, Hróbjartsson A, Nejstgaard CH. Adherence to the PRISMA-P 2015 reporting guideline was inadequate in systematic review protocols. J Clin Epidemiol. 2022;150:179-187. doi:10.1016/j.jclinepi.2022.07.002

2. Nejstgaard CH, Shamseer L, Chan A-W, et al. Updating the PRISMA-P reporting guideline for systematic review protocols. OSF. April 10, 2024. https://osf.io/2znc5/

3. O’Brien BC, Harris IB, Beckman TJ, et al. Standards for reporting qualitative research: a synthesis of recommendations. Acid Med. 2014;89(9):1245-1251. doi:10.1097/acm.0000000000000388

¹Cochrane Denmark & Centre for Evidence-Based Medicine Odense (CEBMO), Department of Clinical Research, University of Southern Denmark, Odense, Denmark, mengmose@health.sdu.dk; ²Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark; ³Department of Medicine, Women’s College Research Institute, University of Toronto, Toronto, Ontario, Canada; ⁴Centre for Reviews and Dissemination, University of York, York, UK; ⁵National Institute of Public Health, University of Southern Denmark, Odense, Denmark; ⁶Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; ⁷Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia; ⁸Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada.

Conflict of Interest Disclosures

An-Wen Chan, Kerry Dwan, Asbjørn Hróbjartsson, David Moher, Matthew J. Page, Larissa Shamseer, Lesley A. Stewart, and Camilla H. Nejstgaard are members of the steering group for the update of the PRISMA-P 2015 reporting guideline. All other authors declare no conflicts of interest related to financial interests, activities, relationships, and affiliations, including employment, affiliation, funding and grants received or pending, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. An-Wen Chan and David Moher are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract.

Funding/Support

This project did not receive any external funding; therefore, design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation were not influenced by any funder or sponsor.