Trial by error

In June, Dorothy Bishop decided to take matters into her own hands. Seething over the inaccuracies in a press report about a study in her field, the professor of developmental neuropsychology at the University of Oxford decided it was time to launch a new prize for science journalism. Via the medium of her blog (http://deevybee.blogspot.com), she called for nominations for her Orwellian Prize for Journalistic Misrepresentation.

"I am offering a prize each year for an article in an English-language national newspaper that has the most inaccurate report of a piece of academic work," she explained. Bishop added that the prize - to be awarded each January - would consist of a certificate and statuette and be based on a points system where errors would be judged against publicly available documents: three points for a factual error in the title; two points for one in the subtitle; and one point for the body.

"I think it stirred things up," she says, adding that although she has yet to receive a nomination, she has had many messages from fellow scientists who say it is a great idea.

"I just thought that this is so typical of what tends to happen ... we have really got to name and shame the people who do this."

Bishop explains how reporters subsequently had told her that they don't write the headlines, but she is unrepentant: the headline is what "screeches" at you, she says; writers should check them and editorial practices should change to make this easier.

"I find it weird that journalists don't seem to have any control and don't actually seem bothered by it," she says. "When you write journal articles, you are told that the headline is what people read."

"Most scientists probably do feel that they get a terrible deal through the mainstream press," says Andy Williams, Research Councils UK research Fellow in risk, health and science communication at Cardiff University. Yet while inaccuracy, distorted facts and misrepresentation may be the most common criticisms scientists make, they are far from the only ones. In fact, there is a litany.

As well as being baffled by the arbitrary nature of what can and can't be covered, mainstream science reporting stands accused of engaging in "kill or cure" sensationalism, failing to acknowledge consensus, whipping up controversy, "dumbing down" science and shoehorning it into a form that perpetuates ideas of novelty and breakthrough that are very far removed from how the discipline actually works.

Yet with impact on the agenda, scientists are also wondering how to engage with the media to a greater degree. So to borrow from Lord Rees, former president of the Royal Society, who used the terms in jest at one "scientists meet the media" party, how should the "nerds" work with the "reptiles"?

Williams attributes much of the bad feeling that exists to a "disparity of interests". The "news values" that drive journalists - such as the need for conflict and newness - are very different from the values and motivations of scientists.

"Scientists don't understand that it is not the job of journalism to be a science communicator. It is the job of journalism to tell a story to sell a paper or gain a bigger audience: that is a basic fact of life, but it's also the root of a lot of bad feeling.

"So many of the things that scientists complain about in the reporting of science stem from the fact that information in the news media is not primarily for the public good. It is about turning information into a commodity to be sold in the market. That is the cause of most of the problems in one way or another ... I don't think scientists will ever like what the media do: they have a different set of motivations."

Alice Bell is a science-communication lecturer at Imperial College London. On the question of inaccuracy, she asks whether the reality is black and white.

"When you look at accuracy, it tends to be quite a personal thing about whether you think something is inaccurate or not, how inaccurate it is and how upsetting it is.

"I think journalists should feel free to re-articulate something from the way it has been articulated to them by specialists. Sometimes people may say that this is not as precise as the original, but that is the nature of mediation - and journalism."

The news article that led Bishop to launch her prize was published in The Observer at the end of May, headlined "Fish oil helps schoolchildren to concentrate". Its errors were so egregious that it became the subject of one of "bad science" critic Ben Goldacre's weekly columns in The Guardian.

Goldacre is much loved by scientists: Bishop describes him as a "hero" and Goldacre himself admits he has "accidentally become a mouthpiece for a small army of disenfranchised nerds". He is admired for his willingness to hold individual journalists to account and to take aim at the "systemic" problems of bad reporting that he sees (he doesn't worry too much about the framework of news values). He also advocates an additional strategy: that parallel to mainstream journalism, scientists should communicate directly to the public through the internet.

His arguments run like this: inaccurate science reporting can have serious consequences for public health. Plenty of mainstream science reporting is fine but some of it is badly broken. He can't quantify the amount because he has not done a study, but the risks are such that people need to bang the drum. Specialist science and health correspondents are less likely to offend, but there is no guarantee that they won't.

But his criticisms have increasingly riled science journalists. While they may think that Goldacre provides a vital if uncomfortable service for their profession, they have been turned off by his unremitting focus on bad reporting and his lack of acknowledgement of the good. They also resent his tendency to generalise about quality from a few bad examples, believing that it undermines their profession.

In the fish oil case, the health editor of The Independent, Jeremy Laurance, fought back. Sick to the back teeth of Goldacre's attacks, he wrote a column that ripped into the Bad Science author for "pistol-whipping" the reporter of the piece, Denis Campbell. Laurance claimed Goldacre had misunderstood the role of journalism: it should offer accurate stenography rather than truth-telling, he suggested, adding yet more fuel to Goldacre's fire.

"Most disinterested observers think standards (of health and science reporting) are pretty high," noted Laurance, citing a report that deemed the profession to be in "rude health".

The report, Science and the Media: Securing the Future, was produced for the Department for Business, Innovation and Skills in January by a group led by Fiona Fox, director of the Science Media Centre, which promotes the views of scientists in the mainstream media. The report's conclusion was based on the fact that both scientists and journalists felt positive about their engagement, with increases in both the number of scientists engaging with the media and science reporters, plus a greater appetite for science stories within newsrooms. But it did not consider the quality-related issues Goldacre has been agitating about, adding that the huge popularity of his "brand of media criticism" meant that there was already an unprecedented level of debate.

Goldacre hit back at Laurance, saying his criticism was legitimate, fair and well evidenced. And Fox entered the fray with a blog post defending Laurance and attacking Goldacre's tone.

Her message to scientists about the mainstream media, she explains, is that the glass is half-full rather than half-empty because so much is now in the hands of specialist, rather than generalist, journalists who cover the field better. Her worry is that Goldacre is putting scientists off from engaging.

"The problem is those scientists who are already so hostile to the media ... all they hear (from Goldacre) is 'don't engage'. There is something at stake here. Ten years after GM, scientists must keep in there."

But problems do not get fixed by pretending they are not there, Goldacre says. He also points the finger at an "old guard" of science communicators, who want to maintain the dominance of the traditional model of scientists being reliant on the media to communicate their work. And he thinks it was a shame he was not consulted for the BIS report.

"They've missed some important points and misread some important criticisms," he says.

As current president of the Association of British Science Writers (ABSW), Natasha Loder has watched these debates with interest. She finds Goldacre's messages "valid, fascinating and sometimes disturbing", but notes that he self-selects bad examples to write about.

"He has a bad science brand, he is never going to say that everything is OK. His book is Bad Science, not Mostly OK, Sometimes Great but Occasionally Awful Science," she says.

Her point is that the "view from the gutter" is never going to give you a "view of the street": "It is like judging the quality of policing by looking at a few bent coppers." Similarly, looking from the top - the approach the BIS report took - is never going to portray an accurate picture of overall quality.

"I don't mean to be evasive, but how does one judge the state of science reporting in general?" Loder asks. "What quantitative measures do we use, rather than the polarised opinions of people with axes to grind?"

What she does know is that science journalists are interested in improving the quality of what they are producing and points to a sell-out ABSW conference last month. "Would political reporters or education correspondents come together to debate whether they should have covered a story better?" she asks.

Simon Lewis is a climate-change scientist from the University of Leeds who has had his fingers badly burned by the mainstream media. He won a victory this June when he used the Press Complaints Commission to force an apology from The Sunday Times for a news article on the so-called "Amazongate" story in which his views were not represented fairly or accurately.

This decision followed on the heels of another PCC victory for a scientist - Aaron Sell, a psychologist at the University of California, who forced the retraction of an article that misrepresented his research from the website of the same paper.

Despite his ordeal, Lewis will not stop talking to the press. "If scientists don't engage there will be much less informed comment," he says. However, he notes the importance of being wary.

"You need to know who you are talking to, what their story is, what they are hoping to get out of it and then exercise the appropriate level of caution," he advises.

To improve the relationship between scientists and the media, Lewis would like to see two things. The first is a "one-stop shop" where scientists can go to find out about particular journalists - including evidence of problems with their reporting in the past and their "political angle". This would help decide the appropriate level of caution to take when dealing with them, he explains. Second, he would like more media training for scientists to help them deal with hacks on the attack.

He notes how relationships between scientists and journalists have shifted in his area since the "Climategate" incident, with some journalists becoming much more aggressive.

"You just have to wise up to that new political reality," Lewis says. "(Journalists) want to push on much more political points rather than science. And I think we have to accept that certain branches of science have potentially major policy implications and therefore are going to be under immense scrutiny.

"That is fine, but scientists have to have some level of confidence in journalists that what they have told them is going to be reasonably faithfully reported - and that is what broke down in my case."

Mark Henderson is science editor of The Times. He also believes the way forward is for scientists to be more discerning about outlets and journalists. As he sees it, there are "demonstrably different" attitudes to science reporting across the mainstream media, and the "biggest mistake" scientists can make is to treat them as a whole and not discriminate.

"Just as there are scientists who fake their data and conduct unethical practices, and it would be unreasonable to tar everyone with the same brush, it is equally unreasonable to tar all journalists with the same brush because some are badly behaved.

"There are some newspapers that are very concerned about getting things right as far as possible while still presenting accessible reporting for a general audience - which is entirely possible to do - and there are other media outlets that may take a different approach as to how they attempt to sell themselves."

Henderson personally wants to "get better" and "be right". "I am happy to engage with constructive criticism of my work, just as scientists are," he says.

He explains the "self-interested" reasons for his desire for accuracy. More broadly, he wants people to feel that they can trust his paper so they buy it. More narrowly, a journalist is only as good as their sources, he says, and a reputation for accuracy and trustworthiness goes a long way.

He takes his hat off to scientists who come up with sensible, constructive ways of engaging when mistakes are made - because they will be made. Science journalists cannot be experts in everything.

He recalls being invited for a lesson in particle physics by a scientist who had picked him up on some mistakes he had made in a story. "Not only did I get better, but it built a better relationship with the scientist, which meant I could easily ring him up."

The "message" that he wants to express to scientists is yes, be wary, yes, be critical, but be aware that there is also a clear way through the mire.

"There is a lot of good reporting, there is a lot of bad reporting ... It's about discriminating between the two. I think scientists can discriminate and actually help the best to thrive by supporting them," he says.

America's Goldacre takes on the PR hegemon: 'It's definitely not journalism'

The US' closest equivalent to Ben Goldacre is Curtis Brainard, a journalist at the Columbia Journalism Review. He intrepidly runs its Observatory website, which since January 2008 has put the science press under the spotlight.

He takes bad science reporting to task in the Goldacre mould, but also critiques the extent to which it is critical, and covers news from the science media industry.

Issues and concerns in the US and UK collide in many ways, he believes. The quality of reporting worries US academics, too (although in the US it is television rather than newspapers where more bad science lurks). Scientists there are also communicating directly to the public online.

As in the UK, a lot has been done to "improve and fortify" the lines of communication between scientists and journalists over the past few years. But unlike the UK - which has the Science Media Centre - this development has been more ad hoc.

In the US, climate change has been the main driver in attempts to improve scientific communications, whereas in the UK, worries over genetic modification and the MMR vaccine have been largely responsible.

"Climate change coverage really spiked in the US from 2005 to 2007 and there was a lot of concern about journalists quoting sceptics and not really understanding the consensus," he explains.

In America, there has also been a major decline in the traditional news media, which has hit science desks hard. "Newsrooms just don't have the means to pay for well-trained science journalists," Brainard says.

Public relations have filled the vacuum, with news sites such as Futurity (bankrolled by research universities) and Science360 (bankrolled by the National Science Foundation) providing coverage.

The NSF has always underwritten some science journalism, he says. But whereas previously it did so via grants to news organisations to produce their own content, now it produces its own. Much of the US News and World Report's science coverage, for example, comes directly from the NSF.

"It's definitely not journalism," says Brainard.

Triumph of the nerds? Bloggers may save science journalism

It is always worthwhile knowing your enemy (or your new best friend), right? So what should scientists know about the mainstream science media?

If the job of science journalism ranges from reporting basic research in an interesting, engaging and entertaining way to taking a critical, watchdog-style stance (the stuff of all journalism), then in practice it gets it in the neck from both sides (and that is after leaving the vexed issues of accuracy and news values to one side).

Ben Goldacre isn't just critical of bad science in the mainstream news, but boring science, too. Alongside other bloggers such as Martin Robbins (The Lay Scientist) and Ed Yong (Not Exactly Rocket Science), he reckons a lot of "interesting stuff" is being missed.

Robbins goes even further. "The best thing that could happen to a lot of science journalism is for it to die a quick death," he says.

The mainstream media, obsessed with the outdated idea that science has to be articulated to people who aren't interested, has "dumbed down" so much that there is little worth reading for those with science backgrounds who might want to "nerd out" with a bit of depth or context, the basic argument goes (though arguably a wealth of popular science magazines such as New Scientist and BBC Focus target just this gap).

Stories in the mainstream media plucked for their supposed "wow" factor are often either "pointless and stupid" puff or invariably about "genes (done uninformatively), space or dinosaurs", Goldacre says (although he acknowledges that the online content of these traditional sources is better).

There is, however, a "revolution in science reporting" taking place that is challenging mainstream journalism, Goldacre notes. Growing numbers of "scientists, academics and other varieties of nerd" write - inevitably in more interesting ways - about the studies they have found by way of websites and blogs.

The "huge opportunity" Yong sees is the rise of social media such as Facebook and Twitter, because on such platforms, those who haven't purposefully sought "nerd content" are going to be exposed to it anyway as their nerd friends share the stuff they like.

"The really interesting thing you realise when you get your nerd succour from the internet is what an even playing field there is between mainstream and informal work," notes Goldacre.

His message to the mainstream media is "nerd up" - get scientists to write about the subject because it will be far more interesting that way, with journalists taking on curatorship or editorial roles instead. And he urges more scientists to go online and become producers in the online nerd revolution.

But what about mainstream journalism's watchdog role? How is it faring in holding science to account? The Department for Business, Innovation and Skills report, Science and the Media: Securing the Future, found this was "by far the weakest area of science reporting today".

Andy Williams, whose study Mapping the Field: Specialist Science News Journalism in the UK (2009) fed into the BIS report, sums it up: "Mainstream science journalism probably isn't critical enough, which stems from it not being independent enough."

His study, based on interviews with specialist correspondents, found them to be increasingly overworked and overreliant on public relations operations to provide the press releases, briefings and sources they needed. As a result, they were becoming "stenographers to strong news sources" rather than "intelligent critics".

"It used to be the case that journalists had time to go and find critical stories, whereas now they don't. They are relying on other people to provide their material," he says.

As Williams sees it, this has had "potentially catastrophic consequences" for the ability of the media to hold science to account. He urges scientists to care "as citizens" about informing the public, but acknowledges that from their self-interested perspective, this might be a pretty rosy place to be: not only is it easier to do a job without media criticism, but there is less room for error if journalists simply cut and and paste press releases.

"The scientific establishment would understandably like to have as much control (of the media) as possible and because of these structural weaknesses it is able to have more control than it probably has ever had," he says.

Fiona Fox, director of the Science Media Centre, is the first to admit that she thinks science journalism can be too referential and she takes her hat off to those who are prepared to take a more critical line. But if journalists want to eat off the SMC's plate, she is certainly not about to stop them. "Let's be honest - structural problems (in journalism) are an opportunity for PR offices."

And it is not hard to see why Fox may be keen to have stories covered by specialist science reporters rather than generalist journalists. Not only are there questions of accuracy, but the former could also potentially be more sympathetic.

"From the very first sociological studies into science journalism, it has been suggested that at times specialists get too close to their news sources and lose their critical edge," says Williams' report.

Yet it is the job of science journalism not only to talk about science but also its social, ethical and political consequences.

Jonathan Matthews runs a small organisation called GM Watch. He explains how in 12 years of campaigning against genetically modified technology, he has contributed to a fair number of science-related articles, but not one was written by a science correspondent from a mainstream paper, even when the issue seemed ideal for them to address.

"Do I think that the majority of UK science correspondents are far too close to the scientists they cover? Definitely. I also have the impression that they're not just lacking in critical distance from the science establishment, but are also almost hermetically sealed off from the rest of us."

The science bloggers hint at an interesting experiment. What if the problem at one end of the spectrum of science journalism could be the solution at the other? What if science journalists curtailed their uninspiring output of basic research news and left it to the bloggers instead? Imagine the free time that would leave for more thorough investigations and more critical journalism ...

"If you spend all your time doing stuff that suddenly a huge number of people are also doing - some better and most for no money - why not redirect your resources?" asks Yong.