Peer review under the microscope in new journal

A new journal has been launched to collect hard evidence on some of the most vexing issues plaguing the scientific process, from conflicts of interest to the failures of peer review.

Research Integrity and Peer Review, due to be launched on 3 May, contains studies exploring topics such as gender equality in research, transparency, and the dissemination of retracted findings through the scientific literature.

Maria Kowalczuk, one of the editors-in-chief of the journal, said that it was designed to scrutinise every stage of the research process: the award of grants; the design and execution of experiments; how scientists write up their results; and finally, peer review and publication.

“There is a lot of opinion published, blogs for example”, about problems in science, she said, “but really there [are] very little data on which to base this. We definitely need much more.”

Existing research into topics such as scientific reproducibility are currently published “all over the place” in different journals, she added, and the aim of the new journal was to bring it all together.

Disputes have raged over the effectiveness of the current system of peer review, with some journals experimenting with “open” peer review where the identities of reviewers are known to all.

Other platforms such as F1000Research have gone even further, subjecting articles to basic checks on intelligibility and ethics before opening them up to online peer review after publication.

Research Integrity and Peer Review is owned by Springer Nature, one of the commercial publishers that have come under fire for perpetuating the existing system of peer review.

Last year, Richard Smith, former editor of the British Medical Journal (BMJ) wrote in Times Higher Education that “peer review persists because of vested interests. Absurdly, academic credit is measured by where people publish, holding back scientists from simply posting their studies online rather than publishing in journals.”

Asked whether the new journal’s ownership presented it with a conflict of interest, Dr Kowalczuk emphasised that the company was using open peer review in some journals. “We are innovating,” she said.

Yet as for the more radical model of post-publication peer review at F1000Research, there were currently no plans at Springer Nature to “go that far”, she said.

The new journal will start off with open, prepublication peer review, the “potential irony” of which has been acknowledged in a blog by Liz Wager, a co-editor-in-chief. “For all its flaws, the current system seems the best place to start,” she writes.

But Dr Kowalczuk said the journal would be open to switching to a post-publication peer review model if enough evidence were collected to support such a move. “If there is evidence from several different studies it would probably be persuasive,” she said.

More broadly, she said that simply amassing evidence of problems in the scientific process would not, on its own, lead to change. “It needs to start with the evidence to know what the problems are and how large the problem is.”

“But then all the stakeholders need to work together” to reshape the incentives and pressures faced by academics, she said. 

david.matthews@tesglobal.com

How retracted papers still echo even after being withdrawn

One of the papers in the first edition of Research Integrity and Peer Review shows just how bad the scientific system is at stopping retracted results from spreading through the global network of research.

Two researchers based in the Netherlands looked at what happened after a Nature paper on sirtuins, a type of protein, was retracted in February 2014.

All but two of the 57 studies in circulation that directly cited the retracted paper simply showed no awareness that it had already been retracted, they found. This was despite the fact that most of these citing papers were published well after the retraction.

At worst, the dissemination of previously retracted results could end up harming patients, says "Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from, the journal Nature".

The most famous example is the discredited link between measles vaccination and autism, which even after retraction still deters parents from having their children vaccinated, it says.

The study confirms previous evidence that retraction does little, if anything, to halt the spread of a paper’s ideas.

“Where one report finds that citation rates drop by approximately 35 per cent after retraction, another report finds no significant decrease in citation rates after retraction,” the new study’s introduction points out.

Harm Nijveen, co-author and a researcher at Wageningen University and Research Center, told Times Higher Education that although many papers cited retracted findings, those article's own conclusions were not invalidated because they did not depend on the retracted research.

However, the spread of retracted findings could do far more damage to scientific knowledge in other circumstances, he warned. “If a finding ends up in a review [of literature], it’s very difficult to get it out” because it was hard to retract reviews, he said.

He said that he wanted to see publishers allow researchers to update their papers more flexibly so that they could easily remove retracted research.

The results also showed that publishers were failing to double-check that submitted papers did not cite retracted papers, he said, something that could be done automatically.

There is still no central database of retracted papers, the new paper points out, although the blog Retraction Watch, along with the Center for Open Science, plans to create one.

David Matthews