Higher education’s access tsar has criticised institutions that spent £50 million on financial support for students without checking whether it had any impact.
The Office for Fair Access found that one in five English universities and colleges (21 per cent) failed to carry out any evaluation of the effectiveness of their outlay on bursaries, fee waivers and hardship funds during 2014-15.
This means that about £50 million was spent without any assessment of whether it made students more likely to enrol, reduced the chances of their dropping out, or led to improved grades.
Offa has called on universities to spend less on bursaries after a study found that, nationwide, they had no observable effect on retention.
Among the institutions that did not carry out evaluations, financial support accounted for up to 92 per cent of total spending under their Offa-approved access agreement, although in some cases it was as low as 13 per cent.
A further 25 per cent of providers reported that they had evaluated their spending only by examining students’ reactions or opinions, as opposed to investigating whether it led to changes in behaviour.
Les Ebdon, the director of fair access to higher education, said that institutions needed to demonstrate that they were not wasting money.
“It is not acceptable to be spending that amount of money in access agreements and not evaluating it,” Professor Ebdon said. “We want universities to properly investigate the impact of financial support.”
Offa’s first review of institutions’ evaluation activities, published on 14 July, says that while some universities had found that financial support could have a positive impact on retention, there remained a “significant proportion of institutions reporting that financial support had no impact”.
Overall, 14 per cent of institutions said that evaluation of their wider outreach and student support programmes was at an advanced and embedded stage; this number had doubled since 2013-14.
Just over half (51 per cent) said that they were undertaking active evaluation and were looking to improve, while 30 per cent said that their work was in development or at an early stage. Only 1 per cent of providers reported that they had not undertaken any evaluation of any kind for access activity in 2014-15.
Universities were significantly more likely to report that they had a well-developed evaluation programme than further education colleges.
The report says that providers with more advanced evaluation programmes were more likely to be making positive progress on the targets that they had agreed with Offa.
“This shows that we have made good advances on evaluation since we started to push this really hard in the guidance we give to institutions,” Professor Ebdon said. “But we are concerned that there are still some providers which are not evaluating their programmes as strongly as they should be.
“Given that they are spending £725 million a year [under their access agreements], it is still a bit of a surprise that we are not seeing stronger evaluation in institutions of how effective that spending is.”
Offa is currently working with the Sutton Trust to develop tools for universities and colleges to use to evaluate their outreach activities, and issued the first tender for the project last week.
