Abstract
Authors Self-Disclosed Use of Artificial Intelligence in Research Submissions to 49 BMJ Group Biomedical Journals
Isamme AlFayyad,1 Maurice Zeegers,2 Lex Bouter,3 Helen Macdonald,4 Sara Schroter4
Objective
Recent surveys report that a high percentage (50%-76%) of researchers use artificial intelligence (AI) in their research.1,2 In April 2024, BMJ Group mandated submitting authors to disclose and describe their use of AI.3 We analyzed the frequency of self-disclosed use of AI in research manuscripts submitted to 49 BMJ Group biomedical journals and identified types of AI tools used and the tasks they assisted with. We also compared the characteristics of manuscripts disclosing AI use compared with those not disclosing it.
Design
A cross-sectional analysis of original research submitted between April and November 2024 was conducted and reported using the STROBE reporting guideline. Main study outcomes were frequency of self-disclosed use of AI, types of AI tools used, and tasks they assisted with. Characteristics of submissions (region of submitting author, number of authors, acceptance rate, impact factor, general medical/specialty journal, peer review model) were extracted from journals’ manuscript tracking systems and websites. χ² test was used to compare proportions of these characteristics between submissions disclosing and not disclosing AI use. Factors associated with disclosing AI use were assessed using binomial logistic regression. A P value threshold for significance was set at .05.
Results
There were 25,114 eligible submissions from Asia (13,505 [53.8%]), Europe (6523 [26.0%]), North America (2795 [11.1%]), Africa (1196 [4.8%]), Oceania (708 [2.8%]), and South America (387 [1.5%]). A total of 1431 of 25,114 submissions (5.7%) disclosed AI use. The most common types of AI tools used were generative AI chatbots (812/1431 [56.7%]), writing assistant tools (182 [12.7%]), and visual/image processing tools (38 [2.7%]) (Table 25-1013). The majority of submitting authors who disclosed AI use reported using it to improve the quality of their writing (1248/1431 [87%]), and 28% (n = 399) reported using it for other purposes, including translation (107 [7.5%]), analyzing or collecting data (44 [3.1%]), generating data and output (87 [6.1%]), code writing (15 [1.0%]), image processing (36 [2.5%]), literature searches (49 [3.4%]), and managing references (8 [0.6%]). The percentage of submissions with disclosed AI use varied significantly by region (highest in Europe: 31.0%; lowest in Oceania: 1.0%; P < .001). Submissions disclosing AI use had a lower mean number of authors (8.39 vs 8.90; P = .005). Authors from Europe (odds ratio [OR], 3.61 [95% CI, 2.11-6.18]), North America (OR, 2.30 [95% CI, 1.32-4.02]), South America (OR, 4.93 [95% CI, 2.62-9.28]), Asia (OR, 2.87 [95% CI, 1.68-4.89]), and Africa (OR, 3.34 [95% CI, 1.92-6.08]) were significantly more likely to disclose AI use than those from Oceania (P < .05 for all). Conversely, each additional author reduced disclosure odds by 1% (OR, 0.99 [95% CI, 0.97-0.99]). Other characteristics of the journal (acceptance rate, impact factor, general medical/specialty journal, and peer review model) were not associated with AI use disclosure.
Conclusions
The percentage of submitted articles with self-disclosed AI use was significantly lower in this study than what has been reported in recent surveys of researchers about their general use. Submitting authors may be underreporting their use of AI.
References
1. How are researchers responding to AI? Oxford University Press. May 23, 2024. Accessed September 1, 2024. https://corp.oup.com/news/how-are-researchers-responding-to-ai/
2. Insights 2024: Attitudes Toward AI. Elsevier. 2024. Accessed September 1, 2024. https://www.elsevier.com/en-gb/insights/attitudes-toward-ai
3. Macdonald H, Abbasi K. Riding the whirlwind: BMJ’s policy on artificial intelligence in scientific publishing. BMJ. 2023;382:1923. doi:10.1136/bmj.p1923
1Research Center, King Fahad Medical City, Saudi Arabia, ialfayyad@gmail.com; 2Department of Epidemiology, Care and Public Health Research Institute, Maastricht University, the Netherlands; 3Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Department of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, the Netherlands; 4BMJ, London, UK.
Conflict of Interest Disclosures
This research is part of an ongoing PhD collaboration between The BMJ and the team at Meta-Research at Maastricht University (UM) on the responsible conduct of publishing scientific research. The BMJ is published by BMJ Group, a wholly owned subsidiary of the British Medical Association. UM is a public legal entity in the Netherlands. This study is part of Isamme AlFayyad’s self-funded BMJ/UM PhD. No exchange of funds has taken place for this research project. All authors express their own opinions and not necessarily that of their employers. Helen Macdonald and Sara Schroter are full-time employees at BMJ Publishing Group. No other disclosures were reported.
Additional Information
The protocol of this research study was registered in Open Science Framework (osf.io/uvqms).