1 Citation   Views   Downloads

Potential bias in peer review of grant applications at the Swiss National Science Foundation

View preprint
RT @eduardo_liveira: Peer reviewers are four times more likely to give a grant application an "excellent" or "outstanding" score rather tha…
"Author-nominated reviewers rate papers more favourably than do referees picked by journal editors" by @snsf_ch. #peerreviewbias https://t.co/imljmFi8b1 Preprint available
Peer reviewers are four times more likely to give a grant application an "excellent" or "outstanding" score rather than a "poor" or "good" one when they are chosen by the grant’s applicants. https://t.co/ABH2Tt1DMr https://t.co/0IZR7zJzPn
Potential bias in peer review at SNSF https://t.co/iBXaH5uebY via @PeerJPreprints applicants recommend referees for their applications, appraisals x4 more likely to be rated excellent / outstanding. hencey many funders only select 1 suggested reviewer from the proposed reviewers
36 days ago
https://t.co/nhYn2JDldt https://t.co/PEaL4gt9oe
https://t.co/IU4GRw6wSM
RT @SantiagoSchnell: Is this study truly measuring bias? https://t.co/YARo9hBqSD To be biased, the "friendly reviewer" will need to exercis…
Is this study truly measuring bias? https://t.co/YARo9hBqSD To be biased, the "friendly reviewer" will need to exercise prejudice in favor of one grant compared with another, usually in a way considered to be unfair. The meta-analysis cannot control this!
37 days ago
This preprint was noted by Nature today... Potential bias in peer review for research funding: https://t.co/1Zwkg2w4aw
37 days ago
RT @NicoleBarbaro: "Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female app…
RT @NicoleBarbaro: "Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female app…
RT @NicoleBarbaro: "Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female app…
RT @NicoleBarbaro: "Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female app…
"Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female applicants" -- https://t.co/6x4JWdyD0p Preprint notes that after relevant confounding variables are controlled, effects decrease sig https://t.co/LyUyrL7VeJ https://t.co/xJwarS6e9M
Interesting, but that's not why I am posting this. The reason? Nature is discussing a paper that was published as a preprint in #PeerJ! Nice... Here's the PeerJ Paper. https://t.co/JZ6OMATeqp ‘Friendly’ reviewers rate grant applications more highly https://t.co/mcFZave4mX
A preprint of the paper is available at https://t.co/iLflO7Y8xT
RT @annasvrn: Applicant-suggested reviewers rate grant applications more favourably: @DalmeetS for @NatureNews on our study of the @snsf_c…
RT @annasvrn: Applicant-suggested reviewers rate grant applications more favourably: @DalmeetS for @NatureNews on our study of the @snsf_c…
@suzan @mikofLohr Voeg daar ook nog aan toe: Recent op Twitter: Peer review niet echt betrouwbaar: https://t.co/cm3XdPEBd1 Recent in PeerJ: Peer review biased: https://t.co/vPicGJomsj
RT @annasvrn: Applicant-suggested reviewers rate grant applications more favourably: @DalmeetS for @NatureNews on our study of the @snsf_c…
37 days ago
Potential bias in peer review of grant applications at the Swiss National Science Foundation [PeerJ Preprints]. From ⁦@annasvrn⁩ and ⁦@eggersnsf⁩ https://t.co/R5vnHO4GIO
38 days ago
RT @annasvrn: Applicant-suggested reviewers rate grant applications more favourably: @DalmeetS for @NatureNews on our study of the @snsf_c…
Applicant-suggested reviewers rate grant applications more favourably: @DalmeetS for @NatureNews on our study of the @snsf_ch peer review of research grant applications. Read the preprint here: https://t.co/HlaPmAGXRK @eggersnsf @rambujo @rachelhey3 @scigeist https://t.co/YPJR4WsSRm
RT @eggersnsf: Out in @PeerJPreprints: Analysis of 38250 peer review reports of 12294 @snsf_ch grant applications by @annasvrn, João Marti…
@MarkLautens @KirstyDuncanMP @cdavidnaylor Related: SNSF released an interesting analysis on PeerJ not long ago: https://t.co/roM4ELTZVx
NOT PEER-REVIEWED
"PeerJ Preprints" is a venue for early communication or feedback before peer review. Data may be preliminary.

Supplemental Information

Table 2. Unadjusted and adjusted differences in scores assigned by external reviewers of grant applications submitted to the SNSF

Notes. Unadj. Effect = unadjusted, crude effect; Adj. Effect = adjusted effect; SE = standard error; 95% CI = 95% confidence interval; *p < .05 (t-test. t-values and degrees of freedom reported online at www.git.io/fhaJx).

DOI: 10.7287/peerj.preprints.27587v2/supp-1

Figure 5: Average evaluation scores by research topic for female and male applicants and proportions of female and male applicants by research topic

Upper panel: Horizontal lines indicate Wald confidence levels, with the number of reviews for each point.

DOI: 10.7287/peerj.preprints.27587v2/supp-2

Figure 6: Average evaluation scores by age group for female and male applicants and proportions of female and male applicants per age group

Upper panel: Horizontal lines indicate Wald confidence levels, side by side with the number of reviews for each point.

DOI: 10.7287/peerj.preprints.27587v2/supp-3

Figure 7: Average evaluation scores by type of affiliation for female and male applicants and proportions of female and male applicants by affiliation

Upper panel: Horizontal lines indicate Wald confidence levels, side by side with the number of reviews for each point.

DOI: 10.7287/peerj.preprints.27587v2/supp-4

Figure 8: Average evaluation scores by nationality for female and male applicants

Horizontal lines indicate Wald confidence levels, side by side with the number of reviews for each point.

DOI: 10.7287/peerj.preprints.27587v2/supp-5

Figure 9: Average evaluation scores by application call deadline

Horizontal lines indicate Wald confidence levels, side by side with the number of reviews for each point.

DOI: 10.7287/peerj.preprints.27587v2/supp-6

Additional Information

Competing Interests

AS, JM, FD, AJ are employed at the Swiss National Science Foundation. JM is seconded at the European Research Council. ME is the President of the National Research Council of the Swiss National Science Foundation.

Author Contributions

Anna Severin prepared figures and/or tables, authored or reviewed drafts of the paper, approved the final draft, investigation, Project Administration.

João Martins conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, prepared figures and/or tables, authored or reviewed drafts of the paper, approved the final draft, investigation, Project Administration.

François Delavy analyzed the data, authored or reviewed drafts of the paper, approved the final draft.

Anne Jorstad authored or reviewed drafts of the paper, approved the final draft.

Matthias Egger conceived and designed the experiments, authored or reviewed drafts of the paper, approved the final draft.

Data Deposition

The following information was supplied regarding data availability:

Associated data is available at Zenodo.

DOI: 10.5281/zenodo.2592509

URL: http://doi.org/10.5281/zenodo.2592509.

Code is available at URL: https://zambujo.github.io/snsf-peerreview/.

The published data is a slightly modified version to ensure that it is anonymized: the information on the date of the call has been deleted, the age variable has been regrouped in 5 year groups and two very special cases have been deleted.

Funding

The authors received no funding for this work.


Add your feedback

Before adding feedback, consider if it can be asked as a question instead, and if so then use the Question tab. Pointing out typos is fine, but authors are encouraged to accept only substantially helpful feedback.

Some Markdown syntax is allowed: _italic_ **bold** ^superscript^ ~subscript~ %%blockquote%% [link text](link URL)
 
By posting this you agree to PeerJ's commenting policies