At a time of crisis, universities must make best use of their limited resources. In the case of the UK, some people have suggested that the 2021 research excellence framework be postponed by a year, as so many things have been. In my view, it would be better to ditch it entirely – and the teaching excellence framework with it.
I am a long-standing critic of both the REF and the TEF, mainly on the grounds that they take up a disproportionate amount of time and energy of academic staff relative to their benefits. It is, of course, all very well to say we should ditch them, but the question then is what to put in their place.
To answer it, we have to consider what these frameworks are trying to achieve.
The REF has a long history, having developed since the 1980s as a transparent means of allocating block grant research funding to higher education institutions. Over the years, it has become increasingly complex and detailed, and has also suffered from mission creep, being used also to incentivise various types of research activity and institutional behaviours. Attempts to simplify it have always been resisted by academics themselves, who insist on a peer-review process in preference to metrics.
Unfortunately, the alternative that we have arrived at has many unforeseen negative consequences, including a huge drain on the time of academics who serve on REF panels, a need for more administrators, and a distortion of incentives that has led to a focus on the production of “world-leading research” and a consequent devaluation of more specialised research activities, and of teaching. HEIs now spend literally years not just on the REF itself but on “mock REFs” that start well before the submission. By definition, not everyone can be “world leading”, yet those who are not producing 3* and 4* papers are made to feel that their work has no value.
The outcome of the REF is a set of rankings of institutions in terms of “research excellence”, which now takes into account not only research outputs, but also ratings of impact case studies and the research environment. The results can be reported solely in terms of average quality or via a “power” measure that combines both quality and volume of staff submitted.
The latter approach is used to allocate research funding, and if that is all you are interested in – rather than allowing universities to use REF rankings to boast about their “quality” – then you would generate much the same distribution if you were to base it on the number of full-time equivalent research-active staff at the institution. I suggest we should do that in the next funding round and ditch the REF 2021.
A counterargument is that institutions will game the process by parachuting in research-active staff just for the REF. But that is readily dealt with by basing the allocation on staff who have been on the payroll for a minimum period, say three years, and/or recent staff who have guaranteed employment for such a period.
Turning to the TEF, its aims are mixed. On the one hand, it is supposed to counteract the tendency for institutions to devalue teaching – a direct consequence, I might point out, of the REF. On the other hand, it is supposed to provide useful information for students who are seeking out a course of study.
Much as I dislike the REF, I regard the TEF as far worse; it was hurriedly thrown together with little consultation and relies on inappropriate statistical analysis of indirect, invalid proxy measures of teaching quality. It then proceeds to turn these into a three-point scale of gold, silver and bronze. Since, for most students, it is not helpful to have a rating that is based on the whole institution, there is pressure to create a subject-specific TEF, but that would compound the statistical problems with the underlying measurements, because they would be based on even smaller numbers. An independent review of the TEF has been carried out but has not been made public. Many of us suspect it has been suppressed because the methodological issues that plague the TEF are unanswerable.
So what’s my alternative to the TEF? Two things. The issue of devaluation of teaching in universities is related to growing reliance on short-term contracts for teaching staff. This simply needs to stop: students need to be taught by people who are treated like valued employees and to whom the institution shows some commitment.
Regarding the second goal of giving students information, the Unistats website offered by the Office for Students provides exactly this, in a manner that allows them to select a course that matches specific features that suit them. That is far more useful than a gold, silver or bronze ranking that just bundles together a host of different, and often unreliable and invalid, metrics into a global score.
Many have pointed out the shortcomings of REF and TEF. Now is the time, as the system approaches breaking point, to accept that they are doing more harm than good to the sector.
Dorothy Bishop is professor of developmental neuropsychology at the University of Oxford.
