Subpanels' lack of experience will put REF credibility at risk, Hefce told

The membership of the 36 subpanels was announced last month by the Higher Education Funding Council for England.

The law subpanel contains just one person who has experience of working on the research assessment exercise, which preceded the REF. None of the Hispanists on the modern languages and linguistics subpanel has prior experience.

Catherine Davies, head of the School of Modern Languages and Cultures at the University of Nottingham and chair of the Iberian and Latin American languages subpanel in the 2008 RAE, said that at least one of the three Hispanists needed experience.

"When you are dealing with such a complex system, with the setting of such crucial criteria and working methods, with constant assessment requiring very fine judgement, all against strict deadlines, inevitably those who have experience will have a better understanding of the process," she said.

She said Hefce had asked that two-thirds of nominees have experience of the RAE, which is being replaced by the REF as the mechanism for allocating about £1.1 billion in annual quality-related research funding.

"If this is how Hefce proposes to keep to its own criteria, how can we trust the credibility of the REF?" Professor Davies asked.

Philip Swanson, head of the department of Hispanic studies at the University of Sheffield and president of the Association of Hispanists of Great Britain and Ireland, said the issue also highlighted the problem of ensuring that the subpanel possessed enough expertise in each language. Hefce made the decision to amalgamate the seven separate subpanels that assessed modern languages and linguistics in the final RAE.

Languages such as Polish and Portuguese are not represented on the REF subpanel.

Professor Swanson said the REF risked losing the "grudging acceptance" earned by the RAE. "If people think their work is not being read by proper specialists in their field, that is when the exercise risks losing credibility," he said.

Graeme Rosenberg, Hefce's REF manager, said subpanels had been selected "to ensure they include an appropriate range of expertise...to develop the criteria for assessment", with the size of each one reflecting the "scale and diversity" of its remit.

He said that subpanels could co-opt extra members if they felt there were "substantive gaps" in their expertise, and would be able to recruit "assessors" to help with assessment, set to take place in 2013.

Professor Swanson said it would be difficult for people who were not full subpanel members to develop a feel for the different star ratings, while experienced figures might refuse to undertake unrecognised "spadework".

But Dr Rosenberg said assessors would play a "full and equal role" with panel members in assessment and would be trained alongside the subpanel members.

He said Hefce had advised that at least a third of subpanellists should have RAE experience and at least a third should not.

"In the case of the modern languages panel, around half of the members have RAE experience. In the case of the law panel there were insufficient nominations of people with RAE experience, and the...members have a range of other qualities and experience that fully qualifies them for the role," he said.

paul.jump@tsleducation.com