Popular beat may drown out genius

The 'audit society' rewards those scientists who dance to its tune, says Peter Lawrence. Too busy working? Two left feet? Too bad

It is fun to imagine songwriters being assessed as scientists are today. Bureaucrats would rank their songs by which radio stations played them in the first two weeks after release. The songwriters would soon find that writing junky Christmas tunes and cosying up to top DJs advanced their careers more than composing proper music. It is not so funny that in the real world of science, dodgy evaluation criteria, such as impact factors and citations, are dominating minds, distorting behaviour and determining careers. Indeed, these measures have now grown into monsters that threaten science. Already, they have produced an "audit society" in which scientists are forced to put meeting the measures above trying to understand nature and disease.

Scientists are assessed according to the impact factors of the journals they publish in. Of course, there is some correlation between quality of work and the impact factor of the journal that publishes it, but this measure has many faults. Crucially, impact factors are distorted by positive feedback - many citations are not based on reading the paper but by reading other papers, particularly reviews. Consider the 48 citations of one article I co-authored in 2002: only eight were appropriate to what was actually reported, three were plain wrong and 37 were incidental. Thus citations may generally be determined more by visibility than by content. Now there is a new trend: scientists are ranked by a single number, the "H index", that counts their papers and the citations they receive. In consequence, I predict that citation-fishing and citation-bartering will become major pursuits.

These measures have had many effects. First, it is now very common for people to gatecrash their names on to author lists for papers whose contents they are largely a stranger to, perhaps by providing a reagent, through horse-trading between group leaders or by the misuse of authority or power.

Second, it is now vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial. This fact has cut a swath through scientific thinking, turning our thoughts away from scientific problems and towards the process of publication. Grisly stories of papers bouncing down a cascade of journals are now the main dish of scientific discourse. It is not unusual for a scientist to spend as much as a year trying and retrying to get something published in a "vanity" journal. This is a massive waste of time and energy that, even so, can bring career rewards. Therefore, I would like granting agencies to investigate the time that leaders of the groups that they fund spend on this paper chase.

Third, trying to meet the measures involves changing research strategy: it is best to follow fashion and work in crowded halls - if you venture into the unknown, you risk interesting no one, and publication will be difficult.

Fourth, scientists learn to hype their work, making a story more simple and sensational by ignoring or hiding awkward results.

Fifth, these measures push people into larger groups, all of whose papers are authored by the group leader - the more people, the more papers. However, I suspect that an increased proportion of young scientists fail in large groups but, because no account is taken of this, such failures do not make a group leader look less productive. In an attempt to increase the number of papers, students are being treated more like technicians and so miss out on learning how to become researchers.

Sixth, it becomes essential to go travelling, to network and build tacit webs of mutual support among colleagues, some of whom will review your papers. It is no wonder that many successful scientists spend bizarre amounts of time touring.

Seventh, the struggle to survive in modern science has acted against modest and gentle people of all kinds - yet there is no evidence that less pushy people are less creative. As the less aggressive people are predominantly female, it should be no surprise that there has been little increase in the representation of women at the top.

The main villains are fashion, the management cult and the politics of our time. People have lost sight of the primary purposes of institutions, and a growing obsession with internal processes has driven more and more bureaucracy at the expense of research. But scientists of all ranks are also to blame, as we have meekly allowed this to happen. We need to fight back, raise awareness of the problems and make changes locally. For example, appointment committees should remember that they are not hiring a number, but a person with a mix of abilities, of which originality is the most important. We need a code of ethics and a means to enforce it, especially with regard to publication. A public discussion on what justifies authorship would be a good start.

There are also difficulties with the assessment of manuscripts: anonymous referees who murder papers for gain should be held to account. One possible approach might be for the large granting agencies to set up an ombudsman, to whom those wronged by maladministration could appeal. It is time to help the pendulum of power swing back to favour the person who works at the bench and tries to discover things.

Peter A. Lawrence is a researcher in the zoology department at Cambridge University and the MRC Laboratory of Molecular Biology. This column is adapted from his article in the latest issue of Current Biology .