If teaching needs fixing, is TEF the right tool for the job?

How do we solve a problem like suspect teaching quality?

It’s a question that was posed to a series of vice-chancellors, teaching and learning experts, union bosses and student leaders who trooped into Committee Room 6 in Parliament today.

The MPs on the Business, Innovation and Skills Committee are digging into the government’s plans for a teaching excellence framework – the details of which were sketched out in the recent higher education Green Paper.

For once, I was on the other side of the fence, giving evidence to the committee, rather than reporting on it, because the committee wanted to understand how established rankings system deal with teaching (and, if you’ll permit a little immodesty, the MPs noted that the Times Higher Education World University Rankings are now the “gold standard”).

For us at THE, it’s vital that our ranking looks at teaching and learning alongside the mainstay of research performance (we also look at universities’ international outlook and links with industry), because we’re ranking universities as rounded institutions, not simply as research institutes.

But there’s no doubt that teaching excellence is a much harder area of activity to measure – or, indeed, define – than research excellence. A comparison I’ve used before is with the US Supreme Court’s struggle in the 1960s to define what constituted obscene material. The justice in the case concluded that he might not be able to come up with a definition, but “I know it when I see it”. We probably all feel the same about great teaching: we know it when we see it, but how do we define and measure it?

Read more: Drop link between TEF and fees, vice-chancellors urge MPs

In the case of the THE World University Rankings, our focus is on the learning environment, which measures a basket of metrics related to the Humboldtian idea that research-led universities (which constitute the world’s top 800 institutions) should be providing a learning environment that is intimately linked to the research effort. These include the doctorate-bachelor’s ratio and the ratio of doctorates awarded to academic staff.

We also use measures such as institutional income, staff-to-student ratio and reputation – which despite being qualitative is an important part of the mix – that all contribute to a fertile learning environment – but do not measure what the teaching excellence framework has set out to measure, which is the actual quality of teaching at a granular level.

Another difference is that the THE World University Rankings focus on the world’s top 4 per cent of institutions – 800 out of about 20,000 globally – and as such tend to focus on universities with similar missions as research-led institutions.

The danger of applying a “one-size-fits-all” basket of metrics to a national system is that there is great diversity within that, diversity that is of huge value and which needs to be preserved, and there must be a risk of universities being led down a path to homogeneity if a small set of (flawed) metrics are employed and directly linked to the financial reward of a higher tuition-fee cap. It’s interesting that a united front appeared to be opening up among others giving evidence to the committee today that the TEF should be decoupled from permission to raise fees if the worst of the unintended consequences are to be avoided.

These are, of course, very complex issues and this barely touches on the questions of those perverse incentives, other metrics that might be developed (such as student engagement and learning gain), and whether or not it’s right that a greater focus is brought to bear on university teaching quality (although the consensus is that done the right way, this would indeed be beneficial).

Some of these topics were discussed in greater detail in the BIS evidence session, which you can watch in the video of the session here (the relevant session starts at 10.08am).

For our part, we’re very proud to be the only world ranking that measures the learning environment, and helps to guide university strategy and performance among the world’s top institutions not only in research, but across all their key missions – teaching and learning included.