REF panellists must be absolutely clear: quality in the UK is not in decline

Universities' expectation that future funding formulas will take into account only work rated 3* ("internationally excellent") and 4* ("world-leading") means that many are planning to submit only those outputs they believe fall into these categories to the 2014 REF.

But Adam Tickell, pro vice-chancellor for research and knowledge transfer at the University of Birmingham, told the Higher Education Policy Institute's annual research conference last week that it would be very difficult for panellists to avoid making relative judgements between submissions and to "drop" some submissions into the 2* ("internationally recognised") category.

This could give politicians and the public a false impression that the volume of high-quality research had declined since the 2008 research assessment exercise.

Bruce Brown, chair of main panel D, which covers the humanities, said this was a concern the panels shared.

But Ann Dowling, the chair of main panel B, which covers the physical sciences, pointed out that panel members' wide experience as peer reviewers meant they would bring a sense of how submissions fitted into the overall spectrum of research quality in their disciplines.

David Sweeney, director of research at the Higher Education Funding Council for England, which administers the REF, said he would "reinforce" the message that panels must make absolute rather than relative judgements.

But Professor Tickell warned that it would have to be repeated "early and often".

The REF panels are currently finalising their criteria and working methods following a consultation this autumn. Final versions will be released in January.

Professor Dowling said there was "unprecedented cooperation and communication" between the main panels, and that every effort was being made to meet respondents' calls for even more harmonisation of practices and terms used - not least to "prevent universities thinking there is something deeper" in terminological differences between panels.

She said respondents had also called for more clarity about the interplay between the two criteria - "reach" and "significance" - for assessing impact case studies.

She confirmed that doing very well on just one criterion would be sufficient for a high score.

She also clarified that it was not necessary for the person who did the underlying research to be submitted to the REF. She said the panels were thinking about how to assess the research underpinning the claimed impact, which is required to be at least 2* in quality.

"It would be great if institutions could come up with ways to convince the panels [that this threshold is met]," she added.

Professor Brown said the panels would try to be more precise about the "threshold criteria" for double-weighted submissions and the way multi-author papers would be treated.

He also insisted that panels' practice corresponded with official policy: "It is an urban myth that they don't stick to the guidance," he said.

paul.jump@tsleducation.com.