Examples of bad peer review and why it is damaging to researchers

Peer review prior to acceptance and publication can make a substantial contribution to the development of research, and reviewer comments can help identify gaps in the literature cited, shortcomings in the methodology and interpretation of data, limitations in the claims being made, and help with communicating the research more clearly. 

Alongside the important gatekeeper role played by editors, it remains arguably the best available approach for the quality assurance of new research. Yet this is peer review at its best and it can often fall short of this ideal. 

My colleagues and I are exploring the quality and usefulness of comments provided by peer reviewers. Although there are ongoing efforts to highlight peer review comments, our approach is to focus more closely on the contribution the peer review process can make to the development of knowledge. 

We began by collating examples of the perplexing comments some authors have received. 

For instance, some reviewers express the opinion that they simply like – or dislike – a paper. This would seem unhelpfully subjective. We encountered comments such as: “I have read this paper several times, and was prepared to like it...but I ultimately did not like it very much at all”; “the interpretation of findings is a bit irritating”; “this paper was ultimately a disappointment”; or, on a more positive note: “I liked this paper”. 

Such throwaway comments can leave authors unclear about what needs to be done to revise and improve their work. 

The questionable clarity of some comments can also be an issue for authors: “I suggest the authors avoid talking about things they do not know enough [about]”; or “the authors are making a completely wrong argument” are particularly unhelpful, while other comments we found were quite opaque or elusive – including, for example: “How about a passing glance at Bourdieu?” 

The tone of comments can also be a challenge. 

For most authors, the time taken to write and submit an article for review is considerable, and there is little value in reviewers being dismissive or rude. Comments such as: “the authors are seemingly ignorant of much of the literature”; or: “the section of Ethical Issues is pointless” seem particularly unhelpful. Most damning of all, in my opinion, one reviewer wrote: “Perhaps a different enterprise should have been embarked upon – a different question with different data”. 

Comments on articles that have been revised and resubmitted can also cause consternation for authors. 

As one reviewer wrote: “The author has satisfactorily responded to the comments to the previous draft[...] It is not a very strong paper”. Another author reported how one reviewer declined to comment on a revised paper. 

Peer review can also involve long delays and this can vary considerably between different journals. It has been argued that this can impact significantly on the career development of new academics. Journal editors and reviewers also need to be alert to articles that contain fraudulent material, while there have also been cases of authors anonymously reviewing their own articles. 

It is more important than ever that academic research is accessible, transparent and that it can be verified. Peer review is a constructive and robust part of this process. Indeed, recent research in the UK, which involved a survey of civil servants, found that peer reviewed journal articles are still an influential source of evidence among policymakers. 

We are not dismissive of the commitment and effort of academics who give up their time to contribute to the peer review process. Of course the quoted comments above are out of context, and almost all of the reviews that we examined contained some useful suggestions. Some reviewers provide a very detailed analysis of an article running to several pages. Moreover, we recognise that it can take time for authors to appreciate the value of some reviewer comments once the cloud of disappointment over an article being rejected has cleared.

However, even if well intended, some comments do strike us as not being as direct and helpful as they could be and they can create frustration for authors. Some reviews can just be a couple of dismissive paragraphs. 

More constructive reviews will lead to more constructive responses. This could also help editors make more informed decisions about whether to accept an article for publication, including when reviewers hold different opinions. It may also help editors to communicate their decision to reject an article even when the reviewer comments are very positive. 

Editors, of course, have their own concerns including looking after the reputation and profile of their journal. 

Kingsley Purdam is an academic based in the School of Social Sciences at the University of Manchester. He is a reviewer for a number of academic journals. His work has been improved by the helpful comments and suggestions of peer reviewers. 

If you are haunted by peer review comments and have examples that you have found particularly unhelpful to improving your article – or suggestions for the ways in which reviewer reports can be improved – please email them to peerreviewresearch@manchester.ac.uk.  All submissions will be treated anonymously.