Abstract
Influence of Using a Systematic Review to Justify New Research on Funding Application Score
Jong-Wook Ban,1 Hans Lund,2 Karen Robinson,3 Ida Svege,4 Jan-Ole Hesselberg5,6
Objective
Many have argued that the need for new research should be determined through a systematic examination of existing evidence to maximize its value.1,2 Our study evaluated the outcomes of following such arguments on funding applications. We tested a hypothesis that funding applications justifying new research with a systematic review would receive higher scores than those without such justification.
Design
We analyzed funding applications submitted to Stiftelsen Dam, one of Norway’s largest health foundations, during the 2024 cycle. We used multiple regression to test our hypothesis while controlling for the influence of the following 5 covariables: requested funding amount, principal investigator’s birth year, principal investigator’s gender, project’s health domain, and research methods.
Results
Of 420 applications received, 193 (46.0%) cited a systematic review, 181 (43.1%) referenced a published or unpublished study, and 46 (11.0%) referred to applicants’ own experience as justification for new research. The median requested funding amount was kr3,000,000 ($267,796), with an IQR of kr2,801,000 to kr3,000,000. There were 158 (37.6%) men and 247 (58.8%) women principal investigators, and 15 (3.6%) who did not disclose their gender. The median birth year of the principal investigators was 1976 (IQR, 1968-1982). Funding applications included research projects in mental health (122 [29.0%]), physical health (197 [46.9%]), well-being (42 [10.0%]), disability (18 [4.3%]), and other domains (41 [9.8%]). Qualitative, mixed, and quantitative methods were used in 53 (12.6%), 145 (34.5%), and 222 (52.9%) applications, respectively. The median funding application score was 4.8 (IQR, 4.4-5.1) on a 7-point scale, where 1 represents poor and 7 represents excellent. Applications that cited a systematic review as justification received scores 0.19 (95% CI, 0.03-0.35) points higher than those that relied on the applicants’ own experience (Figure 25-1129). This could have raised an application with an average score by approximately 100 places in a ranking of 1716 applications. Applications that cited a systematic review received scores 0.12 (95% CI, 0.02-0.22) points higher than those that cited other types of evidence.
Conclusions
Our exploratory analysis showed an association between citing a systematic review and higher funding application scores. Our study has the following limitations. First, we could only control for variables available in our data. So, it was impossible to exclude the influence of other potential confounders, such as study design. Second, we evaluated funding application scores rather than funding outcomes. We could not evaluate the funding outcomes due to the very small number of applications receiving funding each year. Because Stiftelsen Dam heavily relies on funding application scores to decide on funding, we used those scores as a surrogate for funding outcomes.
References
1. Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156-165. doi:10.1016/S0140-6736(13)62229-1
2. Lund H, Brunnhuber K, Juhl C, et al. Towards evidence based research. BMJ. 2016;355:i5440. doi:10.1136/bmj.i5440
1Section of Hospital Medicine, University of Chicago, Chicago, IL, US, jban@uchicago.edu; 2Section of Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway; 3Johns Hopkins Evidence-Based Practice Center, Johns Hopkins University, Baltimore, MD, US; 4Nordic Institute for Studies in Innovation, Research and Education, Oslo, Norway; 5Stiftelsen Dam, Oslo, Norway; 6Department of Psychology, University of Oslo, Oslo, Norway.
Conflict of Interest Disclosures
Ida Svege reported being the former Head of Program Development (Leder for programutvikling) of Stiftelsen Dam. Jan-Ole Hesselberg reported being the current Program Manager (Programsjef) of Stiftelsen Dam.