Peer review: should we let the robots take over?

Many academics bemoan the often thankless task of peer-reviewing other scholars’ research manuscripts before they are published in journals.

So what if peer review became an automated process completed by software? It might be a step too far for some, but a new report argues that it is one of the areas that the research community should be thinking about.

Writing in What Might Peer Review Look Like in 2030?, Chadwick DeVoss, founder and president of StatReviewer, an automated platform that offers journals and authors support with statistics, says that technology could enhance and speed up peer review.

He says that some early artificial intelligence technologies are already being used to address certain issues in academic publishing: for example, identifying new potential peer reviewers from web sources; detecting plagiarism; and flagging up occasions when data have been made up, when researchers have used the wrong statistical tests, and when they have failed to report key information.

Automation could help peer review in other areas, too. Systems to verify an author’s identity, to predict a paper’s impact factor and to suggest keywords are also being developed.

“In the not-too-distant future, these budding technologies will blossom into extremely powerful tools that will make many of the things that we struggle with today seem trivial,” writes Mr DeVoss in the report, published by BioMed Central and Digital Science.

But he stresses that fully automating the publishing processes, including the decision on whether to publish a certain manuscript, “is where a slippery slope gets extra slippery”.

Currently journal editors and peer reviewers give the research community an idea of what is important and help to “distil signal from noise”.

“If we dehumanise that process, we need to be wary about what values we allow artificial intelligence to impart,” he says. “Vigilance will be necessary.”

On the other hand, he points out, automation could shrink to milliseconds the time from submission to publication of a manuscript and therefore expedite scientific communication. It could also get rid of human biases, he adds.

“In the end, if science marches towards a more ‘open’ paradigm, the ethics of full automation become less tricky because the publishing process no longer determines scientific importance,” Mr DeVoss concludes.

holly.else@timeshighereducation.com