REF 2014 impact case studies: government policy cited most

Source: Alamy

Need to know: informing government policy was most frequent in the sciences

The analysis of the 6,679 unredacted impact case studies submitted was conducted by a team from King’s College London and Digital Science, a scientific technology firm. It was set to be presented at the Higher Education Funding Council for England’s REFlections conference on 25 March.

Text mining revealed that the most common kinds of impact related to “informing government policy”, “parliamentary scrutiny”, “technology commercialisation” and “print, media and publishing”.

Parliamentary scrutiny is more frequent in the social sciences, while informing government policy is more prominent in the sciences. Impact on the media is most common in the humanities.

The prominence of media could be seen as surprising given that the REF rules made clear that media mentions did not in themselves count as impact.

Jonathan Grant, director of the Policy Institute at King’s College London and leader of the analysis, said that he planned to examine further whether panels had given credit for media mentions despite the rules.

Another area where rules may have been bent relates to impacts within the academy – such as the writing of textbooks – which were officially banned. Despite this, the analysis identifies students as the second most frequent beneficiaries – after companies and ahead of children. But Professor Grant “worried” about the accuracy of that analysis given the difficulty of identifying beneficiaries using text mining.

He was more confident about broader conclusions, such as that the research underpinning impact was typically multidisciplinary (in 80 per cent of cases), and that impact was multifaceted (with 60 separate “impact topics” being identified).

Impact was also widespread. The analysis found every country in the world being mentioned in case studies; the US, Australia, Canada, Germany and France were the most cited non-UK beneficiaries.

Although the research underpinning impact could date back to 1993, the majority had been published since 2008. Professor Grant suspected that this was because universities had been more confident about the quality of recent underlying research – which was required to be rated at least 2*.

Such risk aversion had also been apparent, he said, in a RAND Europe analysis of universities’ experiences of preparing impact submissions, also published on 25 March. But he added that there was no consensus among the universities on how to improve the process.

paul.jump@tesglobal.com