Beware of the fool’s gold in the policymakers’ El Dorado

Politicians are often accused of making social policy according to ideological conviction rather than a sober assessment of what works. But the problem has often been that the depth of compelling evidence just isn’t there.

Now officials, in partnership with academics, are rising to the challenge by embracing big data. They have begun developing “linked datasets” that bring together all manner of government databases to permit individuals’ socio-economic trajectories to be tracked across time. The aspiration is that this will permit much more rigorous conclusions to be drawn about the causes and effects of different social factors and policies.

New Zealand is among the pioneers. Its “integrated data infrastructure” links data on a host of factors, including ethnicity, exam grades, school attendance, mental health, criminal records and number of children – as well as an array of similar information about individuals’ family backgrounds. The government is using this to probe the causes of poor achievement and non-attendance in higher education; one surprising finding is that parental income is not a significant factor – perhaps because of the availability of income-contingent loans.

Roger Smyth, the retired head of tertiary education policy at New Zealand’s Ministry of Education, has called such data “a policy analyst’s El Dorado”, heralding “a quantum step forward in social policy analysis”.

But it also presents ample opportunity for missteps. That is attested to by no less a figure than David Willetts, the former universities and science minister who commissioned the UK’s own foray into linked data, the Longitudinal Education Outcomes (LEO) project. As Willetts recalls in this week’s main feature, his move was prompted by the difficulty of implementing 2010’s Browne report on university finance. This called for student fees to be uncapped, but for an increasing levy to be applied to them to compensate the Treasury for the likelihood that those charged the highest fees would never fully pay them back. But the top universities argued that this would be unfair since their high-earning graduates would pay off their loans anyway. The solution was evidently to vary the levy according to each university’s repayment record – so two academics were commissioned to merge student records with tax and other data.

As predicted, this revealed wide institutional differentials. But it also shed light on the reasons. The main ones were students’ prior attainment, parental social class and subject studied; “there was not a strong institutional effect independent of these factors”, Willetts notes, calling the fairness of Browne’s model into question.

Now, of course, Philip Augar is revisiting the funding of English tertiary education. His report has been delayed by Brexit and could even be shelved if Theresa May, who commissioned it, fulfils her pledge to leave office before the next stage of negotiations. But, if published, it is widely expected to recommend a lowering of fees. Some observers, however, have suggested that the problem is not so much too much student debt as too many students, and have called for those on courses or at universities with low earnings potential to be barred from taking out student loans.

Willetts is adamant that to adopt such a policy, informed by LEO, would be to “take an interesting new dataset and turn it into a tool of a very significant policy directly constraining the options for prospective students”. This is a role that is “quite simply beyond it” because so many factors influence earnings rates, such as occupation and choice of full-time or part-time work.

Indeed, drawing any policy conclusions from the LEO data still requires something of a leap. Willetts, for instance, has a principled dislike of a graduate tax because it would result in some graduates paying back more than the cost of their education, which he does not consider fair. In his view, LEO strengthens his case since publication of the data would incentivise would-be doctors or commercial lawyers to study or work abroad to avoid the tax. But an opponent could easily retort that there is nothing unfair about the most able contributing the most, and that it would be no bad thing to incentivise the spread of UK influence abroad, as well as to reduce the competition for entry to domestic medical and law schools.

In fact, the evidence of Willetts’ tripling of fees (as well as the New Zealand research) suggests that students are less sensitive to the prospect of long years of repayment than opponents feared. That insight is reflected in Willetts’ preferred funding solution: making sure that “the typical graduate pays back the actual cost of their higher education” by abolishing the interest rate on loans and lowering the repayment threshold to its original £21,000 – “which virtually nobody ever complained about”.

Whether it is fair that graduates should take more of the burden of university funding off the taxpayer is, again, a question of principle; some argue, for instance, that it would be better to make business pay more. But Willetts is surely right that wider access to linked data can only be a good thing. While, in a complicated world, facts are often no match for opinion, they can at least help to ensure that principles are translated into policy in less short-sighted ways.

paul.jump@timeshighereducation.com