more reasons to be skeptical of peer review

Nature’s science news website covered a study showing how peer reviewers are mysteriously way more friendly when they have a personal tie to the authors of a grant. From the article:

Peer reviewers are four times more likely to give a grant application an “excellent” or “outstanding” score rather than a “poor” or “good” one when they are chosen by the grant’s applicants, an analysis of Swiss funding applications has found.

The study, at the Swiss National Science Foundation (SNSF), was completed in 2016, and the SNSF acted quickly on its findings by banning grant applicants from being able to recommend referees.

The authors, who are affiliated with the SNSF, posted their results online at PeerJ Preprints1 on 19 March, and in their paper call on other funders to reconsider their funding processes.

“I think this practice should be abolished altogether,” says study co-author Anna Severin, a sociologist who studies peer review at the University of Bern. Other experts are also wary of the problems that author-picked peer reviewers might cause, but some question whether banning them altogether is the right step.

Though not an indictment of peer review per se, it does cast more shadow on the practice. Why? Many fields are defined by dense networks and people who can embed themselves in networks can generate advantage. If you can identify potential reviewers and ingratiate yourself, you can do well.


50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)
A theory book you can understand!!! Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)
The rise of Black Studies:  From Black Power to Black Studies 
Did Obama tank the antiwar movement? Party in the Street
Read Contexts Magazine– It’s Awesome!!!!

Written by fabiorojas

April 24, 2019 at 12:43 am

Posted in uncategorized

One Response

Subscribe to comments with RSS.

  1. Just clicked through and I agreed with the cautionary comments provided in the Nature news article.

    As with any decision making and evaluation process, bias is a given. But… is it not equally interesting to ask: why are evaluators more antagonistic to ideas from strangers than from friends? Who is in a better position to judge the strengths and weaknesses of scholarly output? Can either evaluation truly be considered the “correct” one?

    I would never want to rely solely on author-nominated reviewers, but I don’t see any reason to eliminate them either.


Comments are closed.

%d bloggers like this: