Something is stirring in the perceived hierarchy of higher education institutions within the UK.
New perspectives on performance are emerging as more metrics become available, complementing those that have been around for some time.
The introduction of the teaching excellence framework is already achieving some of its aims by creating a new dynamic that is disrupting perceptions about "quality" and "elites". The forthcoming changes to the teaching excellence framework and the research excellence framework, soon to be supplemented by the introduction of a knowledge exchange framework, should provide further insight into consistency of performance across the traditional core missions of a university.
To illustrate the potential impact of these new measures on traditional notions of quality within the higher education sector, we have constructed a new university league table that combines the quantitative elements of the TEF and the REF.
Combined TEF/REF Ranking
Source: Lancaster University. See bottom of blog for methodology notes.
It is an attempt to show how universities perform from the combined perspective of education and research activities. This analysis clearly demonstrates that the common shorthand of ascribing performance to mission groups and historical reputation is at best outdated in several cases. We would go so far as to suggest that it signals the emergence of a “new elite” of universities.
In keeping with the penchant of observers of the sector to measure "performance" by creating league table rankings, and perhaps playing to the stereotype that senior managers like simplification, our league table straightforwardly combines the most recent REF and TEF metrics.
Despite well-known concerns about the robustness of TEF data, and a recognition by the Department for Education (DfE) that further improvements to TEF can be made, the data underlying REF and TEF are arguably much more robust than using brand references or historical reputations, which are often used as sloppy shorthand for high quality.
Our approach follows that commonly used in other national leagues tables, with our only "free" choice being the weighting of the REF grade point average for intensity on the basis of the total number of staff on academic staff contracts. This choice aligns with our belief that the research intensity of a university should really reflect all staff engaged in the academic endeavour.
This approach has the advantages of being more likely to reflect the engagement students have with staff developing tomorrow’s thinking of their discipline, as well as overcoming game playing with contract changes and/or preventing distortion by universities only having niches of research excellence that are a very small part of the activity of that university.
The league table provides some calibrations points, as the three genuinely world "Champions League Division" UK institutions occupy the top three places. Yet the methodology also challenges what might be perceived as the conventional order of things. The universities of Keele and Coventry find themselves among the top 20. Meanwhile, some recognised world class institutions have depressed positions because of well-rehearsed reasons around weaker TEF performance than the average (e.g. the LSE appearing at 64).
To answer the obvious suspicion that we might construct a league table that deliberately favours our own institution, we should point out that Lancaster has little to gain as we perform well in all three conventional UK league tables, being currently inside the top 10 of all of them. The table we have constructed places us at 8th, below our most recent league table in the Good University Guide of 6th . Institutions performing very strongly in the TEF (such as Coventry) are rewarded with high placings.
Perhaps more interesting than the rankings of particular universities are the trends that this new league table suggests. It identifies those universities that can genuinely combine high quality education and teaching with research intensity. There is an interesting cadre of universities in the top 20 that includes Loughborough, Bath, Surrey and Dundee alongside Lancaster. These medium-sized, campus-based, genuinely research intensive universities are now clearly a key component of the emerging new elite.
Does higher education really need another league table? Perhaps not – but we believe there are good reasons for examining what institutions, stakeholders and students can learn from the new metrics and what they say about quality and performance within the sector.
The changes in the higher education regulatory system since 2015 were designed to help students make well-informed choices; to drive improvements in teaching quality by assigning teaching the same significance as research; and to enable disruption in the higher education market, reducing the emphasis on "a long established track record" and to promote competition through greater transparency and a more "level" and open regulatory system.
The aim of placing ‘students at the heart of the system’ has since become even more important in political and policy terms, with an increased focus on value for money. The HE sector has also just endured a summer of almost unprecedented criticism focusing on value for money, accusations about cartels and the quality of teaching and standards. There is a clear thirst and need for better information on what universities have to offer.
The government is now consulting on the tricky challenge of developing TEF measures at subject-level, as well as taking a more critical look at graduate outcomes, teaching intensity and degree standards. While these developments are undoubtedly important we would urge caution.
Detailed benchmarks on disciplinary and geographical grounds will be needed to prevent students from being misled. At the same time, there is a real opportunity to inform students’ choice beyond institutional brands and highlight specific experiences students most value. Changes to the REF should also shine a light on the true ‘intensity’ of research-intensives and highlight real excellence, wherever it is found.
Despite the breadth of information available, students still have to navigate the shorthand of what a ‘leading’ university is through misinformation and shorthand used by the press, politicians or their school advisors using outdated notions of excellence and elites. Given the DfE’s commitment to the TEF, it is surprising to see that it still uses old-fashioned notions in policy documents and statements, as well as to rank secondary school destination data.
It is time to use new metrics to provide students and parents with a fresh perspective of what an ‘elite’ university is and create a more level playing field than that based on shorthand and historical artefact. The disruption this is providing to our current HE ecosystem should ultimately translate into both higher quality teaching, an understanding of genuine research intensity and which universities deliver on bot
