REF 2014: results by subject

Source: Miles Cole

Download full REF 2014 results by subject

It has taken years of toil and turmoil, but a new landscape of research excellence emerged this week following the grading of thousands of research outputs and impact case studies at an estimated cost of tens of millions of pounds.

After more than a year of nail-biting since last November’s submission deadline, the results of the 2014 research excellence framework are finally published today.

During that interlude, 36 subpanels have pored over more than 190,000 outputs and nearly 7,000 impact case studies written by more than 52,000 full-time equivalent staff at 154 higher education institutions.

Their verdicts will inform the allocation of quality-related research funding – currently about £1.6 billion a year in England alone – for the next six years, according to a formula to be announced in March. Arguably just as importantly, the results will also determine the institutional and departmental pecking orders – at least in research – for that period.

Times Higher Education’s analysis of the REF results by subject reveals that the University of Oxford is likely to see the most lively departmental Christmas parties this year: its submissions have the highest grade point average in 10 of the REF’s 36 units of assessment (see box, below for an explanation of how GPA is calculated). This compares with the University of Cambridge’s four and the London School of Economics’ three. Imperial College London, the top-ranked multidisciplinary institution by overall GPA, is not ranked first in any subject, but scores consistently highly in all units. By contrast, Oxford ranks relatively low in some units, such as theology and religious studies (12th), English language and literature (13th) and, most strikingly, modern languages and linguistics (32nd).

In 2008, Cambridge departments appeared most frequently at the top of the rankings, heading or co-heading 19 of the 67 units of assessment. Oxford headed or co-headed 11.

When judged solely on outputs – the papers, chapters and monographs submitted for assessment in the REF – Oxford comes top in six units, Cambridge and the University of Warwick in four, the LSE in three and Imperial in two.

The range of institutions that can claim to be top for impact – the new element of the 2014 exercise, counting for 20 per cent of total scores – is much greater. It is not uncommon for several institutions in a unit to score a maximum GPA of 4, indicating that all of their submission is deemed “outstanding”. And while predictions that impact could disproportionately favour post-92 institutions have not been borne out (subject tables) reveals that the life sciences have scored very highly on impact, as predicted. Ranked by GPA for impact, the six units of assessment covering the life sciences – headed by clinical medicine – all cluster at the top, followed by most of the physical sciences, interspersed with some social sciences. The arts and humanities mostly achieve lower scores.

Arguably this confirms those disciplines’ fears about their ability to demonstrate impact. However two of the most nervous disciplines – mathematics and philosophy – avoid the bottom (they are ranked 24th and 32nd respectively), while another, Classics, is the top non-science subject for impact, ranking eighth.

Based on quality of outputs alone, which accounts for 65 per cent of overall scores, the ranking of units of assessment is headed by four physical sciences: chemistry; physics; electrical and electronic engineering, metallurgy and materials; and mathematical sciences. Clinical medicine is only joint eighth with the highest ranked humanities subject, Classics. The highest social science, economics and econometrics, is ranked sixth, but most social sciences languish towards the bottom of the ranking. The lowest output GPA is recorded by anthropology and development studies, and by art and design.

Ranked according to their GPA scores in the “environment” category of assessment, which counts for 15 per cent of overall scores, the social sciences and humanities sit near the bottom, with the life sciences clustering towards the top. Four of the six units of assessment that have seen their overall GPA rise the most in 2014 are social sciences, but many units of assessment – including most in the life sciences – are not comparable to 2008 units.

All this means that the life and physical sciences typically score higher overall than the humanities and social sciences on overall GPA (see national profiles). On this all-round measure, chemistry and clinical medicine are tied at the top, followed by biological sciences; public health, health services and primary care; and physics. The highest ranked humanities subject, in 10th, is Classics, and the top-ranked social science is economics and econometrics, in 16th. This is four places lower than the lowest ranked life science, agriculture, veterinary and food science. The unit for sport and exercise sciences, leisure and tourism has the lowest overall GPA, 2.83.

The lowest ranked science, meanwhile, is computer science and informatics, in joint 29th place. It is joint 19th on outputs but is pulled down by rock-bottom scores in both environment and impact. However, Steve Furber, chair of the subpanel and ICL professor of computer engineering at the University of Manchester, was impressed by the quality of impact case studies he saw, which “really conveyed a strong message that there is work with impact going on right across the sector – not just where you would expect to find it in the high-end institutions”.

He is also sceptical that units of assessment can be meaningfully compared: “There was a calibration process for ensuring reasonable comparability between subjects, but it is a bit like calibrating chalk against cheese. Each subpanel evolved its own internal culture, then applied it fairly consistently, but ensuring consistency between subpanels is much harder.”

Dame Ann Dowling, chair of main panel B, which covers engineering and the physical sciences, and president of the Royal Academy of Engineering, believes there is “good comparability” across subjects under a single main panel, but efforts to ensure comparability across main panels were more limited and should be beefed up next time – especially around impact, where judgements are likely to be “less subject-specific” than they are regarding outputs.

Bruce Brown, chair of main panel D, which covers the arts and humanities, and pro vice-chancellor for research at the University of Brighton, also urges caution when comparing national profiles of different units of assessment. He notes that some receive the majority of their submissions from post-92 institutions, while others are dominated by research-intensives.

“However, I am satisfied that if there are differences in the quality profile [of different subjects] in any element of assessment they are a direct result of quality of research that has been assessed, rather than any differences in practices or working methods,” he adds. “The REF has been a very thorough and rigorous exercise that stuck to the criteria and did exactly what it said on the tin.”