Narrow vision

Next year, the research excellence framework will assess the quality of research at higher education institutions across the UK. British-based academics will select their "best" recent outputs for submission, with the outcome carrying considerable weight because it will inform the selective allocation of £1.6 billion of precious funding to their organisations. The REF will consider only four outputs per researcher from the past six years.

A simple analysis based on its predecessor, the 2008 research assessment exercise, provides insights into the quantity and importance of the output that will not be considered by the REF. I asked 20 UK-based scientists for their four RAE research-output submissions. Using Thomson Reuters' Web of Science platform, I calculated the citations to date for each of their outputs from the relevant period (2001-07).

On average, RAE submissions accounted for 25 per cent of the researchers' publications, and 41 per cent of the citations to date of all the publications: in comparison, the presently top four cited papers from that period per researcher account for considerably more, 62 per cent, of citations to date. Similarly therefore, we can expect that the majority of current British science, both in terms of output and academic impact, will not be deliberated on by the REF. Although the submitted papers will over the years be cited more than the average, many academics are producing work they will not submit for assessment that will eventually prove to be their most impactful.

Should UK scientists "strategise" their research to maximise institutional income by reducing output quantity and thus wastage in terms of the REF, focusing instead on the "big four" publications over each assessment cycle that are likely to get high ratings? Such a policy could have detrimental effects: reducing the publication of student dissertations; exacerbating the "filing cabinet" phenomenon whereby non-significant findings are not published; and decreasing submissions to well-regarded journals that are, however, judged by universities to publish articles that will be too lowly rated by the REF.

In contrast, many scientists (and perhaps members of the public, too) believe that those who receive public funds have a responsibility to publish all their data. Interestingly, focusing on fewer but "better" publications would seem to go against the grain of the UK's proud record of producing more science per unit funding than almost any other nation.

Lewis G. Halsey, Senior lecturer in environmental physiology, University of Roehampton