Can post-publication peer review endure?

Source: Getty

On the lookout: critics have suggested that if commentators were made to reveal their identities then criticism may be fairer, more civil and less likely to result in legal action

The legal action launched by a US scientist who claims that anonymous comments questioning his science cost him a lucrative job offer has raised further questions about the potential for post-publication peer review to replace pre-publication review.

Academics’ gripes with prepublication review are well documented. When only two or three reviewers are asked to comment on a manuscript, the potential for acceptance decisions to be skewed by misunderstanding or bias is considerable. Proponents of post-publication review say scrutiny by the whole community increases the chances of errors being spotted and significance being accurately identified. Some even hope that journal publishing – and its vast associated expense – could be dispensed with if there is no longer a need for editors to organise peer review.

Advocates’ current darling is PubPeer, an “online journal club” set up in 2012 by an anonymous group of early career researchers that allows users to comment – anonymously or otherwise – on published papers. The site’s rapid rise to prominence was highlighted earlier this year by the role it played in exposing the two now-infamous and retracted Nature papers by Haruko Obokata, of Japan’s RIKEN Center for Developmental Biology, that claimed to detail a new way to create stem cells.

However, ever since the tendentiously named Science Fraud website – which listed doubts about published papers – was shut down in early 2013 under a barrage of libel challenges, doubts have lingered about whether post-publication peer review is legally viable.

Most pre-publication reviews are never made public, so even one that is highly critical is no threat to a researcher’s wider reputation. Post-publication review, by contrast, is necessarily open, so critical reviews are more likely to elicit a robust response – which, in an increasingly litigious society, could involve lawyers. As Nature recently admitted in an editorial, fear of legal challenge is one reason why it has “concluded that we cannot usually use retraction statements as a means of highlighting wrong-doing” – even when official investigations have reached misconduct verdicts.

Fears of “lawyered up” academics will be strengthened by the case of Fazlul Sarkar, a distinguished professor in cancer research at Wayne State University in Detroit. As first reported by Retraction Watch, he claims that anonymous comments posted on PubPeer this summer led to the withdrawal of a $350,000 (£220,000) a year job offer by the University of Mississippi.

Under US law, PubPeer is not liable for the remarks of its commenters, but Sarkar has subpoenaed it to demand the identity of those he believes to have libelled him – whom he intends to sue. The site responded by taking down the offending comments but said it will resist the subpoena and will cease in future to hold any information – such as an IP address – that might identify its anonymous contributors.

Behind the usernames

Some observers have suggested that the anonymity PubPeer offers is problematic because it offers impunity to those who slip past its moderators and unfairly shred others’ reputations. If commenters were required to reveal their identities, some argue, criticism would be fairer, more civil and less likely to come to a lawyer’s attention.

One proponent of that view is Philip Moriarty. Earlier this year, the professor of physics at the University of Nottingham had a heated exchange on PubPeer with defenders of Francesco Stellacci, Constellium professor at the École Polytechnique Fédérale de Lausanne in Switzerland, over whether Stellacci’s belief in the existence of “stripy nanoparticles” is due to basic errors in microscopy. Moriarty says he was not concerned that Stellacci – who has complained of a witch-hunt – might sue him, but he admits that attitude may just reflect his own “combination of naivety and stupidity”. But however robust discussions become, he believes critics should never retreat behind anonymity, and should always be compelled to identify themselves – at least to moderators. “All you need is one disgruntled postdoc or PhD student and anonymous commenting can be really damaging,” he says.

But, according to an emailed comment from PubPeer, its users have rarely abused their anonymity: “Abusive comments with no substance are both ineffective and easily spotted. Conversely, if a comment makes a valid point, the motivation for posting it is irrelevant. The worst we see is misguided comments made because of some misunderstanding – but, in that case, the authors or other users can explain, and it is probably beneficial for such explanations to be available to all.”

The email admits that anonymous comments are “often the most critical and may highlight features indicative of misconduct”. But it also insists they are important since “the biggest problems matter the most”.

Moriarty concedes that having to reveal their identities even to a moderator could put off vulnerable early career researchers, and he suspects that PubPeer’s popularity – in contrast to some previous experiments in post-publication review – is down to the possibility of anonymity. But he suggests that finding a way to make comments citeable – and, hence, count towards scientific prestige – might warm up some cold young feet.

However, Dave Fernig, a professor in the University of Liverpool’s department of biochemistry, observes that discussion forums that lack anonymity contain “a lot of hagiography” and rarely provide new insight into a paper: “If we lose anonymity, then many will fear to criticise. The consequence is that work that needs questioning will stand and we will have moved from science to faith,” he says.

The UK’s notoriously punitive libel law was recast last year to make it harder for claimants to sue. A defence was introduced for websites that take down offending comments promptly, and peer-reviewed papers were exempted from the scope of the law. However, discourse around papers remains unprotected and the cost of defending a libel action means Fernig’s call for “a good test case” to see where the limits lie seems unlikely to attract willing participants.

However, US libel law is weighed more in defendants’ favour. So while PubPeer accepts that further legal “attacks” are inevitable, it notes that, despite Sarkar’s actions, its contributors continue to be “active”, and it is “optimistic that the robust legal protections for free speech and anonymity in the US will enable sites like PubPeer to defend themselves and their users effectively”.

As for Moriarty, he is crossing his fingers that such confidence is not misplaced: “If you are publicly funded and you put your research into the public domain but no one can criticise you for it without facing legal proceedings, that seems to me to be a very badly damaged system.”

paul.jump@tesglobal.com