João Martins,1 François Delavy,1 Anne Jorstad,1 Matthias Egger1
The Swiss National Science Foundation (SNSF), the leading public research funder in Switzerland, relies on external experts to review grant applications. Applicants can propose reviewers, provided there are no obvious conflicts of interests. On average, applications receive 3 reviews, 1 of which is typically from a reviewer proposed by the applicants. We examined whether the source of the review, the gender of the principle applicant and the reviewer, and the country of affiliation of reviewers influenced the scores given to grant applications submitted to the SNSF.
Reviewers scored applications from 1 (poor) to 6 (outstanding). We calculated mean scores by source of reviewers (applicant vs SNSF), country of affiliation of reviewers (Switzerland vs international), and gender of applicants and reviewers. We fit a multivariable linear regression model adjusting for all these variables plus calendar year of submission, discipline (21 disciplines), and applicants’ age (5 age classes) and affiliation (4 institution types).
Between 2009 and 2015, 36,993 reviewers assessed 12,132 applications for the SNSF. The mean (SD) score of reviewers proposed by applicants (n=8308) was 5.12 (1.01) vs 4.47 (1.25) for reviewers proposed by the SNSF (n=26,594). Mean (SD) scores were 4.19 (1.27) for Swiss experts (n=8399) vs 4.76 (1.19) for international experts (n=26,503); 4.44 (1.25) for female (n=7121) vs 4.67 (1.22) for male (n= 27,781) principle applicants; and 4.48 (1.26) for reviews from female (n=6933) vs 4.66 (1.22) from male (n=27,969) reviewers. In adjusted analyses, the gender differences were attenuated, whereas the other differences changed little (Table). All differences were statistically significant.
Applications received higher scores from applicant-proposed reviewers and lower scores from Swiss-based experts. Scores were lower for applications submitted by female applicants. Our results are compatible with a positive bias of reviewers chosen by the applicant, or a negative bias of experts based in Switzerland, and cannot exclude bias against female applicants. Interestingly, female reviewers consistently scored applications lower than male reviewers, independent of the applicant’s gender. Panels making funding decisions should be aware of these potential biases. Given the association between scores and source of reviewer, the SNSF no longer accepts reviewers proposed by the applicants.
1Swiss National Science Foundation, Bern, Switzerland, firstname.lastname@example.org
Conflict of Interest Disclosures:
The authors are employees of the Swiss National Science Foundation.