When ref fails to blow whistle

Flawed research on the dangers of ecstasy published in Science has led to questions about the quality of peer review, says Anna Fazackerley.

Almost a year ago, the US neuroscientist George Ricaurte captured the attention of the world's media with a paper suggesting taking a few doses of ecstasy in one evening could eventually lead to symptoms resembling Parkinson's disease. The research was published in the internationally respected journal Science , but some scientists in the field were unconvinced. Last month, the story took a new turn when Science printed a retraction. The research team had discovered that, due to a labelling error, the substance they had been administering to monkeys was in fact not MDMA (ecstasy), but methamphetamine (speed).

Science was quick to applaud the team from Johns Hopkins University for "so thoroughly investigating the conflicting data that they had received in their laboratory, and for tracking down the source of the inconsistencies".

Katrina Kelner, the deputy manager for life sciences at the journal, says:

"This is an excellent example of how science itself is self-correcting."

But not everyone in the field is satisfied. And questions are being asked that have resonance for British science.

Several influential figures say they would have rejected Ricaurte's paper if they had refereed it. "I think the doses used and the wholly inadequate way they presented the data was wrong," says Barry Everitt, an expert in drug addiction at the University of Cambridge. "They were trying to argue very strongly that this had resonance for humans using the drug."

Colin Blakemore, who takes over as chief executive of the Medical Research Council this month, raised objections to the paper last year. He is determined that they not be swept under the carpet, and he has written to Donald Kennedy, editor-in-chief of Science , demanding an inquiry.

He says the referees should have been suspicious that a "common recreational dose" of the supposed MDMA proved lethal to a substantial proportion of the monkeys used in the experiment when the proportion of deaths it causes in its human users is far smaller. Police estimate that 1 million people take ecstasy in the UK each year, but only a few die. "If the referees did not spot a problem so glaring that I - and I am not a pharmacologist - noticed, then Science needs to answer questions about the quality of its peer review," Blakemore says.

He also mentions an allegation circulating on the scientific grapevine: that Science had a political motive for choosing to publish Ricaurte's paper. Blakemore points out that it was published shortly before controversial "anti-rave" legislation was debated in the US Congress. "Did they decide to overrule the referees, or not to get the paper refereed at all as it was so topical?" he asks. "I cannot see a charitable interpretation of what has happened from Science 's point of view."

Another factor is that Alan Leshner, the new chief executive of the American Association for the Advancement of Science, which owns Science , used to be the director of the US National Institute for Drug Abuse. Nature - a journal that publishing insiders predicted would show some solidarity with Science - has queried why Leshner took the unusual step of issuing a personal statement endorsing the Ricaurte paper.

Science says suggestions of political motivation are "absolutely baseless and without merit". And Ricaurte dismisses the political conspiracy theories as "silly" and "baffling". He says he had no idea the drug legislation was coming up and says his paper was rigorously peer reviewed.

"My lab prides itself on independence in collecting data and letting that data speak for itself." He adds that people have jumped to conclusions about how his research on monkeys translates to humans. "We were very careful to indicate that studies in humans had not been done."

But whatever the truth on the political front, there is no doubt that the research was seized on by the media and simplified to the extent that people were reading that a night's clubbing could give you Parkinson's.

Blakemore says the retraction has reduced the impact of any accurate research on the danger of drugs: young people just won't take it seriously.

But Everitt, who is a reviewing editor for Science , points out that if the paper had not had relevance to humans it would never have passed muster with Science. "The big journals are looking to publish a paper that will be picked up by the news media. So they want the stuff that is splashy," he says.

He estimates that the journal receives about 1,500 neuroscience papers a year. Because it can publish one a week at most, competition is tremendously fierce.

He explains: "It's great as a scientist to get a paper in Science as the impact factor is monumentally high. So you write the paper with a twist that is attractive to them."

Everitt argues that this may have been the case with Ricaurte's paper and he wonders why the reviewers didn't pick up on this.

Such speculations about who these reviewers were and why they didn't ask the questions other academics feel they should have asked highlight a growing uneasiness about the peer-review system. As Blakemore says: "It is the gold standard, but it is never open to scrutiny."

In fact, such scrutiny is under way in the UK. Amid fears about the impact of increasing competition for funds and rising dependence on commercial funding in particular, the Royal Society and the Sense about Science campaign group have set up working parties to examine peer review.

Interestingly, the Ricaurte ecstasy paper had been singled out by Sense about Science as a possible example of where the system might be going wrong.

Blakemore, who sits on this panel, fears the anonymity of peer review gives people the opportunity to behave in a way that they wouldn't if it were more transparent.

Peter Cotgreave, the director of Save British Science, agrees. "I have a problem with anonymous peer review. If I'm asked I always sign my report, and I think that's the way it should work."

But others are adamant that the system is dependent on the element of secrecy. Everitt says: "I sometimes sign reviews if I know the person and I am being a bit difficult. But this (removing anonymity) would castrate the peer-review process. No one would dare give anyone else a hard time."

The British Medical Journal is moving to make the refereeing process more open. It is running a pilot scheme in which referees' comments, and the author's responses, are published on the BMJ 's website. The results of this trial are expected early next year.

Blakemore has asked Science to follow suit. He wants them to publish the original reviews of the Ricaurte paper, although he points out that they can do so without revealing the authors' names.

"If they are concerned about their reputation, the emphasis is on them to defend it," he concludes. "We need information and a statement from Science that is a little more credible than commenting that these problems could not have been picked up by referees and the scientists should be praised for detecting them."