Everyone’s a critic

Trisha Greenhalgh disagrees with my opinion that pre-publication peer review is a waste of time (“Peer review not yet sunk”, Letters, 4 June). Her disagreement, however, is based on a small technical point on one paper that I mentioned in support of one of the many arguments I advanced. Even if she were right in her one point, it would hardly amount to a refutation of my arguments.

She quotes one of the studies we at the BMJ did that showed that peer reviewers are poor at detecting errors in papers. She says that she was a reviewer in one of the studies and saw so many errors in the paper that she described only three. She says, without any evidence, that she is “sure many reviewers would have done what I did”. As far as I can remember, she’s wrong; and she may be unaware that we did similar studies more than once.

She also says that “the BMJ’s staff – then, as now – viewed peer review as a technical task (‘spotting errors’) rather than a scholarly one (interpretation and judgement)”. I can’t answer for now, but I doubt that that is the case, and certainly it wasn’t when I was the editor. Most of the discussion when peer reviewing the papers was around interpretation, and indeed some of the “errors” that we inserted in the papers were to do with interpretation.

It remains true that most reviewers don’t spot most errors, some of them egregious.

I accept that there are problems in studying peer review, but the case stands that we have substantial evidence of the downside of peer review and virtually none of the upside.

Perhaps Greenhalgh might be able to design some better studies to investigate the value or otherwise of peer review.

Richard Smith
Via timeshighereducation.co.uk