High-flying graduates ‘underestimate skill level’

High-achieving graduates tend to rate their skills and abilities lower than those who leave university with weaker grades, says a study on how learning gain might be measured.

Several pilots are under way at UK universities as part of a £4 million Higher Education Funding Council for England scheme to see if students’ improvement over the course of their degree can be quantified, and possibly included in future guises of the teaching excellence framework (TEF).

However, research by a University of Warwick academic suggests that two of the most established ways to gauge so-called “distance travelled” at university – tracking grade improvement and self-reporting skill development – appear to offer contradictory results on student performance.

Those who graduate with a higher degree classification are also those most likely to self-report little improvement in their graduate skills, such as writing ability, numeracy and computer skills, over the course of their degree, according to the study by Heike Behle, research fellow at the Warwick Institute for Employment Research.

Meanwhile, graduates with lower grades rated their skill set higher than more academically gifted students, even if their actual marks suggested that they were weaker, says the study, whose results were presented at the Society for Research into Higher Education’s annual research conference, which took place in Wales last month.

“As you increase the level of credentials, you see a decrease in graduates reporting their written and computer skills highly,” Dr Behle told Times Higher Education.

Her study used data from Warwick’s Futuretrack survey, which tracked about 20,000 UK students who started university in 2005-06 over six years and asked students to rate their literacy, numeracy and computer skills both at the start of their degree and at the end of it.

The Futuretrack survey also contained information on how well students had performed at university in comparison with their A-level results on entry.

The results may suggest that students with higher grades are actually far more self-critical of their own skill sets than those who perform less well in exams, leading to a lower self-reporting of skill improvement, Dr Behle said.

“When you enter higher education, you are probably not aware of the limitations of your literacy or computer skills, but that awareness increases as you progress through university,” she added.

Although self-reporting of skills could play some role in assessing learning gain, Dr Behle said, it was important to recognise its limitations.

“The main risk is that you expect students to be objective about their level of skill,” she said. “That may be possible in the third year, but it is much harder when students have just arrived at university.”

However, she also urged policymakers to be “very cautious” about using the “very crude” approach to learning gain of simply comparing final degree classifications with Ucas tariff point scores on entry.

“The problem with this method is that if a student enters university with very high tariff points and gains a high class of degree, is it true that they did not improve?” Dr Behle said.

“We also need to be careful about thinking that every metric is comparable across the entire sector,” she added.

jack.grove@tesglobal.com