Cut REF work: Check the best and sample the rest

The research excellence framework should sample only a selection of university departments in each round to minimise costs and reduce the "crippling" workload for assessors, it has been suggested.

The idea has been tabled by John Ellis, professor of media arts at Royal Holloway, University of London, who was a member of the communication, cultural and media studies subpanel for the 2001 and 2008 research assessment exercises.

The REF is to replace the RAE as the mechanism for distributing about £1.5 billion in quality-related research funding in England each year.

Professor Ellis said the results of successive RAEs had shown progressively less variation than their predecessors, so it was reasonable for REF assessors to maintain previous ratings unless new evidence came to light.

But checks should always be run on every top-rated department, as well as on units that thought their performance had significantly improved or declined, he suggested.

He admitted that few of the latter would flag themselves up, so he suggested that checks could be initiated by subpanels on the basis of traditional metrics such as grant income, PhD completion rates and number of staff.

Professor Ellis dismissed suggestions that the 2014 REF would have significantly different results from the 2008 RAE due to it incorporating an assessment of impact, because he did not believe that the Higher Education Funding Council for England would be able to persuade David Willetts to include the measure.

The universities and science minister remains to be convinced that there is a "robust" way of assessing impact that commands wide consent among the academic community.

Professor Ellis said his reforms would reduce the workload of sub-panels by as much as 50 per cent, allowing them to examine each department more thoroughly.

"There was a degree of being overwhelmed by the amount of material we had to look at in the time limit set by the exercise last time," he said.

Institutions that submit their research could also see their workloads and associated costs decline.

"The current exercise is low in cost only because most of its costs are borne by the submitting institutions, including the preparation of submissions, the 'free' labour given by the institutions that employ the panellists, and the loss of the research they could have been doing," Professor Ellis said.

But a Hefce spokesman said a sampling approach had been dismissed by the late Sir Gareth Roberts, president of Wolfson College, Oxford, in his 2003 review of research assessment.

"The multiple purposes of the REF, including research management and public information, as well as the allocation of substantial sums of public money, require us to continue to produce regular and comprehensive quality assessments," the spokesman said.

"In developing plans for the REF in 2014, we are paying close attention to minimising the workload for higher education institutions and panels, including using information that has already been collected for other purposes," he added.

paul.jump@tsleducation.com.