Abstract
Research Culture Influences in Peer Review: A Targeted Thematic Analysis of Current Challenges in Peer Review
Lesley Uttley,1 Louise Falzon1
Objective
This study aimed to identify current challenges in peer review relating to research culture using a targeted thematic analysis of research culture in health and biomedical research from a wider scoping review.1
Design
To understand challenges of peer review influenced by research culture, the literature searches of a wider scoping review1 were subject to a focused qualitative thematic analysis relating specifically to peer review, following PRISMA-ScR (scoping review) guidelines2 and adhering to Nowell et al’s3 trustworthiness criteria for thematic analysis. PubMed and Web of Science were searched for empirical studies published between January 2012 and April 2024 in health and biomedical research culture. Eligible articles for this study were those that reported relevance to peer review (regarding manuscript publication or funding applications) in the title or abstract. Editorials, discussion pieces without data, and articles relating to peer review for conferences were excluded. The main outcome was the qualitative themes from the included articles that were associated with peer review. Relevant papers were subjected to inductive content analyses by 2 authors (L.U. and L.F.), which involved noting key findings to identify recurring themes relating to the research culture topic or finding discussed. The themes were then deductively categorized according to a framework for research integrity across 4 domains: inclusivity, transparency, rigor, and objectivity.1
Results
Fifty-three papers were included from a total of 1680 citations generated by literature searches. Relevant themes (n = 20) regarding the challenges of peer review, ordered by prevalence, were reviewed for thematic content across the 4 domains (Figure 25-0850). Inclusivity included gender biases (n = 8), status bias (n = 2), and diversity of peer review panels (n = 5); transparency included clarity of journal expectations (n = 11), open peer review advocacy (n = 4), and blinding of author characteristics (n = 6); rigor included predatory journals (n = 3), prevention of spin (n = 4), lack of statistical examination (n = 2), potential benefits of artificial intelligence (n = 3), need for training in peer review (n = 3), and potential harms from artificial intelligence (n = 1); and objectivity included willingness/refusal/lack of incentive (n = 4), burden (n = 5), influence of conflicts of interest of reviewers (n = 3), unprofessional comments (n = 1), self-citation bias (n = 2), novelty or positivism bias (n = 2), confirmation bias against innovative research (n = 1), and homophily (preference for shared characteristics of authors) or cronyism (n = 2).
Conclusions
A variety of research culture influences underpin peer review challenges in the traditional academic publishing model. Themes pertain to 4 cornerstones of research integrity1 relating to inclusive workforces, transparency of processes, use of best practice methods, and the objectivity (and incentives) of peer review. Emerging threats from technological innovations in conjunction with incentives fuelled by research culture are likely shaping the research landscape. Sustained and joined-up conversations between authors, publishers, and reviewers about potential peer review innovations are required to safeguard genuine research and aid the dissemination of findings that promise authentic advancements in scientific research.
References
1. Uttley L, Falzon L, Byrne JA, et al. Research culture influences in health and biomedical research: rapid scoping review and content analysis. J Clin Epidemiol. 2024;178:111616. doi:10.1016/j.jclinepi.2024.111616
2. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-473. doi:10.7326/M18-0850
3. Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods. 201728;16(1). doi:10.1177/1609406917733847
1School of Medicine and Population Health, University of Sheffield, UK, l.uttley@sheffield.ac.uk.
Conflict of Interest Disclosures
None reported.
Funding/Support
Lesley Uttley is supported by a grant from the UKRI Medical Research Council (MR/Z504063/1).