Make all see that deadline matters

Frustrated by slow reviewers, Marc Hauser and Ernst Fehr suggest we get tough on tardiness and reward promptness

Every researcher knows the experience: you send a manuscript in for review and it disappears into the ether as you wait a painfully long time for a response. As scientists, we want immediate gratification: either accept or reject. This enables us to move on with our work. Alas, the review process does not work this way and, from our perspective, it seems to have deteriorated over the years. Some people review quickly, others do not. Yet there are no rewards for the swift or punishments for the slackers.

We would like to propose a solution based on the logic of incentives to stimulate discussion, while also appreciating the complexities of incentive provisions. As such, we see this article as an opening card in a game that we hope will help us attain a better equilibrium, one from which an improved system for refereeing emerges.

All journals have requirements for a timely response. Here is a proposed solution to the problem. Whenever a reviewer sends in his or her review, the editors log the date and a positive or negative value that indicates its relative timeliness: negative for those arriving before the deadline, and positive for those arriving afterwards.

People who turn in their reviews late are punished, whereas those who stick to deadlines are rewarded. To make the punishments count and discourage future transgressions, we recommend that for every day since receipt of the manuscript for review plus the number of days past the deadline, the reviewer's next personal submission to the journal will be held in editorial limbo for twice as long before it, in its turn, is sent for review.

To illustrate, consider an individual who is given three weeks to review and responds two weeks after the deadline. Since the total review time is five weeks, his or her next submission will sit in the editorial office for ten weeks before being sent out for review. Journals reward timely reviewers by sending their manuscripts out for review as soon as they come in and, if accepted, by pushing their papers high up into the publication queue.

We suggest some answers to some obvious problems. First, in the case of multi authored papers, only the primary corresponding author should trigger a punishment or reward assignment.

Second, although we realise that the timely reviewer is only gaining what should be the normal state of affairs, it is the cost to the negligent reviewer that is most important. Given the documented benefits of punishment in a variety of co-operation games, we imagine that this policy might well speed up the review process and curtail the number of slackers.

Third, if a given reviewer is the kind of person who rarely gets reviews in on time, he or she could exploit this system by simply refusing to participate or by sending in a less than helpful response, thereby avoiding the costs altogether. To close this loophole, we suggest that for every manuscript an individual refuses to review, a one-week delay is added to reviewing his own next submission. Thus, if a reviewer rejects two consecutive reviews, his next submission sits in editorial limbo for two weeks.

As for unhelpful reviews, these occur even with the present system, and it is an empirical question as to whether the rates would increase under the proposed process. If they did, we would propose some minimal criteria for a useful review; anything less would be subject to the same penalty as proposed for opting out. For reviewers who refuse or turn in insufficient reviews, the only way to break the cycle of penalties is by providing a substantive review for the journal.

Fourth, journals may worry that by implementing this policy they might lose manuscripts from some of the more interesting scientists, who may happen to be slow reviewers. We fully recognise the distinct possibility that there probably is not a strong positive correlation between the quality of scientific research and the timeliness of reviews. But given the hierarchies among journals in every field, as well as the diversity of options, we do not expect this to be a significant problem.

For the proposed system to work, journals must fully commit to this policing policy. Editors may sometimes be tempted to violate it in order to clear the manuscript table, but this will not influence the status of a reviewer. In essence, editors must punish wrongdoers, full stop.

As humans, we are highly sensitive to rewards and punishments - perhaps not as exquisitely as rats in the proverbial Skinner box, but close enough.

Clearly, the review process is broken. It is time to consider a fix. We have proposed a solution based on the logic of economic incentives and the evolutionary origins of human nature.

Marc Hauser is a professor of psychology, organismic and evolutionary biology at Harvard University, US. Ernst Fehr is director of the Institute for Empirical Research in Economics at Zurich University, Switzerland. This article is adapted from a letter to the Public Library of Science Biology open access journal.