REF review ‘will focus on diverse outputs and research culture’

Plans to reform the UK’s research excellence framework (REF) may lead to a radically different exercise in which research culture is valued as highly as outstanding publications, a policy expert has predicted.

Announcing a review of the REF, which is used to distribute about £2 billion in research funding annually, science minister Amanda Solloway focused on the “pressure to publish in particular venues”, which “wrongly suggests that where you publish something is more important than what you say”. She noted that 97 per cent of outputs in the 2014 REF were “text based” and mainly journal papers.

That criticism suggested that the REF review may seek to broaden the type of outputs submitted by researchers, explained James Wilsdon, Digital Science professor of research policy at the University of Sheffield, who expected to see a push to include research datasets, thinktank policy papers, exhibitions and other diverse outputs in the next audit.

“Despite the current guidance that outputs should draw from a diverse set of venues, there is still definitely a strong weighting towards articles in journals with high impact factors,” said Professor Wilsdon, who added that “publication in a prestigious journal gives some reassurance of quality”. “If you’re entering something into a competitive process, you want to be sure that output is a banker – a dataset on Figshare may be more helpful for science, but it doesn’t give you that external assurance,” he added.

One way to address this issue would be to reduce the importance attached to outputs, which have a 60 per cent weighting when peer assessments are made by REF panels, said Professor Wilsdon. Panels also examine the impact of research, which has a 25 per cent weighting, while the environment in which research was conducted accounts for 15 per cent.

“If you had an even weighting of outputs, impact and environment, the REF would start to look like a very different and interesting exercise,” said Professor Wilsdon.

An increased emphasis on environment would move the exercise towards a broader consideration of research culture, team science and research integrity – issues championed by the new chief executive of UK Research and Innovation, Dame Ottoline Leyser.

Professor Wilsdon said Ms Solloway’s speech indicated a desire to complete the decoupling of outputs from individuals recommended by the 2016 Stern review, which led to the requirement for four outputs for every submitted full-time equivalent researcher in 2014 being reduced to an average of 2.5, and a minimum of one output per researcher, in 2021. “If you slice the knot between people and outputs completely and just look at units, that will encourage a more holistic view of research culture,” explained Professor Wilsdon.

David Sweeney, executive chair of Research England, which will oversee the REF review, said that he was “glad that the sector and its partners will have the opportunity to discuss what they really think is excellent research”.

“We are delighted by the work done by our highly talented academics, but we look forward to capturing the support that is around them which helps them to deliver the outcomes of which the UK is so proud,” he added.

However, Tim Softley, pro vice-chancellor for research and knowledge transfer at the University of Birmingham, said he would be concerned about any plan to reduce the importance attached to research published in highly cited journals.

Speaking at a Higher Education Policy Institute event, Professor Softley said that while research culture was important, “we do seem to be getting into the habit of saying there is something wrong about going after excellence”.

“There is nothing wrong with trying to publish a high-quality publication in a prominent journal where lots of people will read it,” said Professor Softley.

jack.grove@timeshighereducation.com