Research intelligence - What's on the cards for the REF?

As academics fret over whether they will be submitted to the research excellence framework - and what will befall them if they are not - heads of research are keeping their cards close to their chest in this evolving game of poker.

Fears that historically low numbers of researchers will be submitted to the 2014 REF have abounded ever since the Department for Business, Innovation and Skills instructed the Higher Education Funding Council for England, at the end of 2010, to confine quality-related funding to “internationally excellent research”.

Hefce and the other UK funding councils responded by removing 2* research, defined as “internationally recognised”, from their funding formula. And although many commentators have argued that 2* is a necessary stepping stone to 3* and 4* research, no one expects it to be reinstated any time soon.

Consequently, as Adam Tickell, pro vice-chancellor for research and knowledge transfer at the University of Birmingham, put it, universities have “no rational reason to submit people who haven’t got at least one 3* piece of work”. And since 2* research was the largest category in most universities’ results from the 2008 research assessment exercise, large numbers of academics fear being deemed “unREFable” in 2014.

Such worries were articulated in a statement released in June by the British Sociological Association. “Whereas [in the] RAE 2008, selectivity tended to be either based upon the requirement of four items meeting the criteria for inclusion, or, at worst, a grade point average [GPA] of 2, it now appears that many institutions are adopting a much more restrictive policy of a GPA of 2.75 or higher,” it said. “This means that colleagues risk being excluded despite producing research of international quality.”

The statement also complained that universities’ internal rating of their research would be “necessarily less rigorous” than that conducted by the REF subject panels, and that its public nature could damage staff morale. It feared that non-selected academics’ jobs could be put at risk during the coming “volatility in student numbers and income”.

How high will they go?

A source told Times Higher Education that some research intensives were planning to impose GPA thresholds as high as 3. THE tried to confirm this by submitting a freedom of information request to all mainstream universities.

Of the 126 contacted, 14 failed to respond, four sent uninformative responses and 20 refused to reveal their figure, mostly on the grounds that disclosure would be likely to prejudice their “commercial interests” by - as the University of Leeds put it - giving others “the opportunity to amend their own preparations in order to influence the outcome of the exercise to their own benefit”.

Another 11 universities planned to impose unspecified variable thresholds, and 32 said they had not yet decided. Only 15 were definitely not planning to impose any thresholds.

Of the 30 universities that provided substantive information about their GPAs, only two - Brunel and Bath Spa - were imposing thresholds lower than 2. Nineteen had settled on higher thresholds and 12 of those expected an academic’s submission to contain at least a majority of work rated 3* or higher.

The latter group included London Metropolitan University - ranked equal 107th in the 2008 RAE, with a GPA of 1.84 and an indicative submission rate of 31 per cent of eligible staff - and Queen Margaret University - ranked 129th, with a GPA of 1.37 and a submission rate of 61 per cent.

This news might be thought to lend weight to fears that, according to Kevin Schurer, pro vice-chancellor for research and enterprise at the University of Leicester, are widely shared among research intensives: namely, that universities with relatively little research above 2* - and so with little to gain financially from the REF - will submit only their 3* and 4* researchers (perhaps fewer than 20 per cent of their total staff eligible for submission) purely in order to maximise their position in the prestigious GPA league tables. He said that a vice-chancellor from a post-1992 university had recently admitted that this would be his strategy.

This would lead to “odd” results that would “give people - especially those outside the sector who do not understand the system - a potentially misleading picture” of where research excellence and critical mass lay, Professor Schurer said.

He added that this was particularly the case because similar tactics were not a realistic option for universities that claimed to be research intensive, where research was embedded in the culture “both as a critical part of promotion and one of the ways in which you acknowledge a person’s contribution”.

High-stakes game

Professor Schurer was sure some research intensives would apply very high GPA thresholds in the REF, but he wondered whether they were sufficiently aware of “what such a policy would mean in terms of volume, projected income, staff morale and equality assurance issues”.

The University of Leicester was ranked 51st in the 2008 RAE, submitting 93 per cent of eligible staff and recording a GPA of 2.45. The top ranked university, Cambridge, submitted 100 per cent of eligible staff and achieved a GPA of 2.98.

Professor Schurer was not surprised that few institutions wanted to reveal their hand for 2014. “It is like a poker game: everybody is trying to watch what everyone else is doing,” he said.

The only Russell Group university that provided THE with unambiguous information about GPA intentions was the University of Birmingham, which will impose a threshold of 2.75, assessed by internal peer review that will be informed, where appropriate, by metrics.

In 2008, Birmingham ranked 26th, with a GPA of 2.64 and an 87 per cent indicative submission rate. Professor Tickell expected a submission rate of 75-80 per cent this time, but emphasised that non-submission would not, in itself, be deemed evidence of poor research performance.

He acknowledged that some teaching-led institutions might concentrate on maximising their reputational ranking, but he insisted he didn’t “obsess” about league tables. “If the University of Poppleton has a GPA of 4 in history on the back of one person, that really doesn’t bother me. But if among the Russell Group and 1994 Group we are way down in the bottom quartile that really does bother me.”

He warned universities aiming to submit only 3* research that they could receive “a bit of a kicking” from the REF panels, whose need to differentiate between submissions, combined with feeling the need to avoid saying all of them were “internationally outstanding”, would inevitably lead them to make ample use of the 2* grading.

He said the REF would be a fairer and more representative - albeit more expensive - assessment of research quality if universities were required to submit all eligible staff. Professor Schurer agreed, adding that an alternative would be for Hefce to release GPA rankings weighted according to the proportion of academics returned.

“I would not blame institutions for being highly selective given the messages from BIS [on 2* funding], but I fear that the outcome may not be to their liking as it may damage both the credibility of the REF and research reputation in the UK,” he said.

paul.jump@tsleducation.com.