Identifying reviewers weakens peer review, study suggests

Scientific journals that require the disclosure of reviewer identities risk making the peer review process weaker and more biased against younger and female researchers, a global analysis has found.

The study was based on more than 4,000 reviewer submissions to a leading ecology journal over more than a decade, and helps affirm the general sense of the research community that anonymous reviews are good practice, said Charles Fox, professor of entomology at the University of Kentucky.

The results showed “a pretty strong gender divide,” with male reviewers more than twice as likely as their female counterparts to voluntarily identify themselves, Professor Fox said.

The comparison also showed that signed reviews had a “quite substantial” tendency to be less critical of the authors, he said. On a rating scale of 1 to 4, the reviews with signed authors averaged nearly half a point higher, he found.

For his study, Professor Fox took advantage of the fact that the journal, Functional Ecology, gives its reviewers the option to publicly reveal their identities.

Over the two periods for which he studied outcomes – 2003 to 2005 and 2013 to 2015 – nearly 6 per cent of Functional Ecology’s reviewers signed their comments to its prospective authors, he wrote in his analysis, published in the Proceedings of the Royal Society B, a UK-based journal specialising in biology.

The norm in academic publishing is for reviewers and their comments to be kept anonymous. Professor Fox said his outcome upholds the general preference of researchers as reflected in surveys – that they accept reformist calls to make their reviews public, published alongside the articles they have judged, but want to keep their own names private.

Professor Fox said he also reviewed a separate data set of article reviews that another researcher had assembled for other purposes, and found that its patterns aligned with his findings from the Functional Ecology reviews.

As with other situations where signed reviews are optional, the Functional Ecology database showed that reviewers often made case-by-case decisions on whether to make their names public. That practice revealed a clear tendency for reviewers to sign their positive assessments and not sign their more critical ones, he said.

The analysis also showed that when a prospective author suggested a reviewer, that reviewer signed the assessment 72 per cent of the time.

Also, reviewers who identify themselves as “professor” are much more likely to sign their reviews as compared to those who use the “Dr” title, which usually suggests a more junior status, Professor Fox said.

paul.basken@timeshighereducation.com