How to build the teaching excellence framework

Jo Johnson, the new minister for higher education, surprised nobody last week by asserting his determination to deliver the Conservative Party’s manifesto commitment to “introduce a framework to recognise universities offering the highest teaching quality”. This will now be known as the teaching excellence framework.

He did not go into a great deal of detail in his speech about what the framework would look like or what precisely the recognition would be for high-quality teaching, although he did indicate that he was open to considering financial incentives and rewards.

In fact, the level of openness and willingness to listen and respond to what the sector thinks was quite striking and positive. I think it is imperative that the sector engage positively with the Department for Business, Innovation and Skills over this, because I’m not sure it is an initiative where a strategy of kicking it into the long grass or hoping for death by a thousand cuts will work.

I think a key consideration in developing the TEF ought to be that it is light touch. I’m sure all universities that participated in the research excellence framework would want to avoid replicating the bureaucratic burden involved in that. The TEF should be based on metrics, ones that already exist or could be gathered with relative ease.

Johnson was also clear about this in his speech when he said: “I have no intention of replicating the individual and institutional burdens of the REF. I am clear that any external review must be proportionate and light touch, not big, bossy and bureaucratic.”

Based on that principle, I think the data should fall into three themes: input, output and peer judgement.

1. Input

By input, I mean the quality of the content and delivery of teaching and wider academic experience of students.

There are a number of data that could be used, or newly collected, to inform this; but I think we have one already in the bag and a couple of others that could be gathered quite easily if we try hard enough.

• Qualified teacher status: the Higher Education Statistics Agency has already been gathering data on this. I know it’s not a measure that the whole sector has united behind yet, but with one more push and some finessing of the Higher Education Academy’s Professional Standards Framework, we could get some sector-wide, comparable data. There will be plenty of wailing and gnashing teeth about this I am sure, but it is notable that this came top of student expectations when the Higher Education Policy Institute surveyed them recently – we should meet that expectation, it won’t do anyone any harm!
• Research impact: I was trying to think of a way of quantifying the fact that students should have the opportunity at university to learn and understand the very boundaries of knowledge and understanding in their chosen subject and to learn from those involved in defining and discovering the new boundaries of knowledge and understanding. The REF scores would tell some of this story, but it occurred to me that the impact scores are probably a more effective and targeted way of measuring the ability to communicate and engage students with world-leading research.
• Enrichment: it is widely accepted that a student’s academic experience is defined not only in the classroom, but also by the co-curricular activities that are available. We are already engaged in an initiative to gather all this data to inform the Higher Education Achievement Record (HEAR). Be it work placements, study abroad, community engagement, leadership of sports clubs or academic representation, these activities improve student academic development and make a significant contribution to an excellent learning and teaching experience. It would take some work to finesse our systems to gather this data, but it can be achieved and should make an especially useful feature of a TEF.

2. Output

By outputs, I mean demonstrable measures of students’ gaining knowledge and skills during their studies. I think there could be three datasets that could be used here, two that we gather already and one that could be derived from existing data.

• Learning gain: this term means many different things to different people. As a school governor, I have become familiar with the concept of “value added”, where a contextual measure of a student’s knowledge, skills and ability is taken on entry and then progression is assessed from that at various points up to a pupil leaving the school. I am not at all convinced that a measure that looks at Ucas tariff through to degree classification is the most effective measure of learning gain, mostly because of all the problems that we know exist with the comparability of degree classifications. I think a measure should be developed that looks at the entry qualifications that students arrive with and then how far along they get with their higher education. That way, universities that recruit students with no or few qualifications could be rewarded and could see the recognition increase as students achieve higher-level qualifications. For example, a student with no qualifications attending a university and achieving a Level 5 HND would have travelled the same distance as a student with the traditional A levels who then achieves a Level 6 BA (Hons).
• Employment data: we currently have the Destination of Leavers from Higher Education survey, which captures reliable data about what students are doing six months after graduation. There is talk of improving this to use HMRC data to show the real earnings of student at one, three, five and 10 years post-graduation, which would be a positive development I think. Either way, this is a key piece of data to show what students are capable of doing and achieving after graduation and should be included in any measure of teaching quality.
• Grades: as long as we have the old honours classification system, the data are not comparable. But if a concerted effort is made to move to the grade-point average system, we might achieve more comparability and be able to develop data that reflect and measure that.

3. Peer review

A purely data-driven approach would not give sufficient space for input from peers. I include students as part of the peer community here because I feel strongly that we should consider their feedback and views of the quality of teaching to be of equal, if not greater, value than that of academic peers because, above all else, students are the only people who can tell us what it is like to be a student.

• The National Student Survey (extended to postgraduates): I’m pleased that the recent review of the NSS appears to have concluded that the survey is immensely valuable and not in need of a radical overhaul. However, it needs refreshing to sort out some of the dated and more nebulous questions (personal development, anyone?!). So this survey and its outcomes should undoubtedly form an important part of the TEF; I would argue that it should be heavily weighted in comparison to other metrics, too.
• External review: I am not advocating a return to subject review or inspection regimes of old; they ran out of steam more than 15 years ago, and everything we learned about them then still stands (burden, bureaucracy, gaming the system, diminishing quality of the “inspectors/reviewers”). But I think a positive judgement from the university’s most recent Quality Assurance Agency institutional review should be a prerequisite for inclusion in the TEF, to demonstrate that core academic standards are being maintained and to show that the teaching is built on solid foundations.

This is an attempt by me, from my perspective, to list what should be included in a teaching excellence framework. Each of the individual sets of data listed above can (and I’m sure will) be picked off on its own and rubbished as an insufficient measure of teaching quality. This may be true: for example, the DLHE only tells you if students get jobs, it doesn’t tell you if they would have got them anyway or whether their degree actually helped them. But nested in with a number of other datasets (some that we gather already, some that we need to start gathering) it can start to piece together a rich picture of the quality of teaching and learning.

This approach of picking apart the minutiae has been very successful for the sector in the past and it may succeed again, but I am doubtful. We have an opening to engage positively and constructively with this initiative and to shape it; if we try to nitpick and shower the whole thing in treacle, I suspect we’ll get the sort of TEF we deserve!

Derfel Owen is director of academic services at UCL. This post was first published on his blog page.