Free app has the cite stuff for REF

The "methodologically indefensible" peer-review element of the research excellence framework should be scrapped and replaced by citation data drawn from a free internet application.

This is the view of Patrick Dunleavy, professor of political science and public policy at the London School of Economics. He told Times Higher Education that the Publish or Perish app, created last year by Anne-Wil Harzing, professor of international management at the University of Melbourne, offered a much easier, cheaper and more robust way to assess research.

"It is mythical to suppose that any group of people can assess 200,000 research outputs," he said. "Everybody knows it, but universities go along with it because they think it is the best way of defending their (financial) position."

Publish or Perish allows researchers to compile citation information about each of their publications over specific time periods. It also calculates a range of other statistics, such as average citation counts and h- and g-indexes.

Professor Dunleavy said the app's use of data from Google Scholar gave it much better coverage of books than other citation databases, making it as relevant to the humanities as to the sciences.

He denied that citations were a less reliable indicator of research quality outside science, adding that recent research identified a strong correlation between citation counts and the outcome of peer-review exercises across a range of disciplines.

Professor Dunleavy said that every academic should be asked to compile their own Harzing profile, which would be checked by their institutions and submitted to the most appropriate REF subpanel. The data could also be made freely available online so that the academic community could provide further oversight.

Analysts from the Higher Education Funding Council for England would then standardise the data - including weightings for researchers' career stage - and produce draft reports for each department and university, Professor Dunleavy suggested. Subpanels would meet only a few times to examine discrepancies and consider special pleading.

He added that the comprehensive picture of UK research output created would also be useful to the academy and the government, because, unlike REF data, it would not be "distorted" by "game-playing" over who was submitted, or by "artificial" limits on the number or nature of admissible outputs.

Following a pilot exercise in 2008-09, Hefce concluded that citation information was "not sufficiently robust" to be used as a primary indicator of quality in the REF. But Professor Dunleavy said Publish or Perish was already being used by hiring committees and he expected "everyone" to be using it by 2014.

paul.jump@tsleducation.com.